Hello Henrik, Thanks Thanks Thanks a LOT!!
Thanks Entire wget team for helping me so generously. Even in your busy schedules. Issue is now resolved. I used Live HTTP headers add on as suggested by you and it really helped me. Initially i passed all the headers as displayed in the capture window, with "--header" option of wget. Later i understood that session details have to be captured. I used "--keep-session-cookies" option of wget, since sessions details have to be saved. I saved the session details in cookies.txt, which i used later to load ADHOC query page and run the query. Below are the commands i used to get the query results. wget -O logon.html --keep-session-cookies --save-cookies cookies.txt --post-data 'loginID=mylogon&password=mypasswd' https://www.collabnet.timeinc.net wget -O query_res.html --load-cookies cookies.txt " https://www.collabnet.timeinc.net/servlets/AdHocQuery?query=select+count%28*%29+from+HELM_USER&Button=Run+query " Special thanks to you and whole team for sparing your time to help me out with the issue. Please let me know if i can be part of your team or i can be of any help to you. HAVE NICE DAY !! With wishes, Bhargavi On Tue, Jan 24, 2012 at 3:20 PM, Henrik Holst <henrik.ho...@millistream.com>wrote: > I looked at your web site and it does not perform standard http > authentication so --username and --password cannot be used to logon to > that page. > > You have to supply the username and password using --post-data there > aswell. If you had followed my advice to use the Live Headers extension > with Firefox you would have seen exactly what to do, please use that > tool aswell as learn some http basics and you will soon learn how to > perform what you want. Because since we do not have access to that site > of yours (no username or password) we as a community will have quite > some hard time to tell you exactly how to proceed since we cannot test > things at our end. > > Anyways, as I wrote I tested to perform a logon attempt with the site > and with Live HTTP Headers extension active I could see that the > authentication should be performed like this: > > wget --post-data "detour=https%3A%2F%2Fwww.collabnet.timeinc.net% > 2F&loginID=username&password=password&Login=Login" --save-cookies > cookies.txt "https://www.collabnet.timeinc.net/servlets/TLogin" > > Replace the "username" and "password" in the post data with your account > details. > > However since I have no account on that site of yours, I do not know if > this really works and whether the detour=xxx thing is really needed and > whether you have to also add a Referer: header or not to the request. > > So if I where you: > 1. Install Live HTTPS Headers extension in Firefox > 2. Open it's capture window > 3. Perform a logon and a sql query > 4. Study the results in the capture window and try to mimic what you see > with wget. > > /HH > > > tis 2012-01-24 klockan 10:11 +0530 skrev Bhargavi N: > > Hello Henrik, > > > > > > I am saving the log on details in the cookies same way as you have > > mentioned. > > > > > > But cookie file looks empty.. > > > > > > Next i loaded the cookie.txt with the > > URL: https://www.collabnet.timeinc.net/servlets/AdHocQuery > > > > > > I saved the results page and then opened in firefox, i get the same > > logon page to ollabnet instead of results page. > > > > > > Steps to run the query: > > > > > > 1) I need to logon to collabnet website: > > https://www.collabnet.timeinc.net > > > > > > 2) In the collabnet website, we have an option to run the query: > > > > > > https://www.collabnet.timeinc.net/servlets/AdHocQuery > > > > > > I need to go to this page and then need to provide SQL query in the > > query text area. > > > > > > Then need to submit the button "Run Query", which will submit the > > query. > > > > > > 3) Next need to download the results page to my local directoy on > > LINUX box. > > > > > > Please help me regarding this. > > > > > > Thanks ! > > > > > > Regards, > > Bhargavi > > > > > > On Mon, Jan 23, 2012 at 12:43 PM, Henrik Holst > > <henrik.ho...@millistream.com> wrote: > > Ok, > > > > most probably the first site (where you logon) returns a > > cookie which you must present to the other site (where you > > perform the sql-query). So > > > > wget --username xx --password yy "https://collabnet website" > > -O /dev/null --save-cookies cookies.txt > > > > That will logon and save the resulting cookie in the > > cookies.txt file. Next you have to send this cookie and your > > query to the other site: > > > > whet --load-cookies cookies.txt "https://the query site" > > --post-data "the query" -O result > > > > And the result should be in the result file. > > > > Now as Angel Gonzales wrote you probably have to send the > > query using the form like --post-data "query=Select%20*%20From > > %20table" where the sql query in question has to be > > url-encoded (something that wget cannot do for you) which is > > mostly replave all spaces with %20 and also all occurences of > > +, &, " and ' with their respective %hex code. > > > > A easy way to see exactly which formfield to use in the > > post-data is to install the Live Headers extension to Firefox > > and open it's capture window and peform the query on the site > > using Firefox, you'll then get to see the exact query, what > > the GET string is, what the post-data is etc. It's a very good > > start to go from. > > > > > > /HH > > > > 2012/1/20 Bhargavi N <bhagc...@gmail.com> > > I meant i am unable to logon with to the website. > > > > > > Sorry, to confuse you with lot of questions. > > > > > > I have to logon to collabnet website first, providing > > username and passwd. > > > > > > Next i want to go to servlet page where i can provide > > SQL query in the textarea of the form. > > > > > > Then i need to submit the form and download the > > results page to my local directory in linux box. > > > > > > I will be running wget command on linux box. > > > > > > Please help me regarding this. > > > > > > Thanks ! > > > > > > Regards, > > Bhargavi > > > > > > > > On Fri, Jan 20, 2012 at 7:20 PM, Bhargavi N > > <bhagc...@gmail.com> wrote: > > Hi Henrik, > > > > > > Thanks for the reply. > > > > > > But when i run as you have suggested me, then > > i get the login page of the collabnet > > website.. > > > > > > instead of the servlet page where i can enter > > sql. > > > > > > Please help me ! > > > > > > I am unable to logon to that page 1st. > > > > > > Thanks! > > > > > > Regards, > > Bhargavi > > > > > > On Fri, Jan 20, 2012 at 4:44 PM, Henrik Holst > > <henrik.ho...@millistream.com> wrote: > > Well, it normally should work with > > this: > > wget > > " > https://collabnet.net/servlets/adhocquery" --user="username" > --password="password" --post-data "select * from emp" -O output_filename > > > > But then according to the man page, > > wget only supports post-data this is > > in urlencoded format like > > "key=value&key2=value2" so wget might > > not support sql queries at all. > > Perhaps cURL works better for your use > > case. > > > > /HH > > > > > > 2012/1/20 Bhargavi N > > <bhagc...@gmail.com> > > Including the team to this > > mail chain.. > > > > ---------- Forwarded message > > ---------- > > From: Bhargavi N > > <bhagc...@gmail.com> > > Date: Fri, Jan 20, 2012 at > > 4:30 PM > > Subject: Re: [Bug-wget] > > Regarding wget to download > > webpage > > To: Dayanand Kumbhar > > <kdayan...@gmail.com>, > > tle...@exelana.com, > > keis...@gmail.com > > > > > > > > Hello Everyone, > > > > I am extremely thankful to > > all of you for the help > > regarding wget. But > > still i am unable to get the > > right page downloaded. > > > > Requirement: > > > > I need to run adhoc sql query > > on remote web page. ie; > > collabnet. The > > webpage has text area where i > > can enter sql query and then > > click submit. > > > > Once i submit sql runs and > > results are displayed in the > > page. > > > > This REMOTE page is call to > > servlet ie; > > > https://collabnet.net/servlets/adhocquery for example. > > > > I want to pass SQL query as > > form data through POST/GET > > method and get the > > sql results page downloaded to > > my local directory on LINUX > > box. I am > > running wget on LINUX host on > > commandline / shell script. > > > > I will invoke wget with the > > URL > > > https://collabnet.net/servlets/adhocquery and > > formdata ie; SQL as > > --post-data "select * from > > emp". Finally i want to > > download the sql query results > > page to my local directory. > > > > I tried all the options that > > you suggested me, but still > > nable to get the > > right page. Every time it is > > displaying me the index page > > for collabnet, > > which i do not need. I need to > > run the servelet page with the > > formdata > > passed to it. > > > > Then i want to get the results > > page downloaded to my local > > directory on my > > linux host. > > > > Please help me regarding this. > > > > Again thanking all of you for > > your reply and helping me out. > > > > Thanks ! > > > > Regards, > > Bhargavi > > > > On Wed, Jan 18, 2012 at 12:40 > > PM, Dayanand Kumbhar > > <kdayan...@gmail.com>wrote: > > > > > Hi > > > > > > i am assuming you have to > > post the query string > > > You can use this > > > > > > wget -qO- > > http://example.com/page.php > > --post-data "SQL QUERY" | cat > > > > > > > > > On Wed, Jan 18, 2012 at > > 12:24 PM, Bhargavi N > > <bhagc...@gmail.com> wrote: > > > > > >> Hello Team, > > >> > > >> My name is Bhargavi and i > > am working on UNIX for over > > 7years. > > >> > > >> I work on project, that > > needs to download results from > > remote web page. > > >> This web page has adhoc sql > > querying option. I need to run > > this sql query > > >> on webpage and get the > > results downloaded using wget. > > >> > > >> How can i do that ? > > >> > > >> Please suggest me way to > > download the results page > > using wget. > > >> > > >> Thanks ! > > >> > > >> Warm Regards, > > >> Bhargavi > > >> > > > > > > > > > > > > -- > > > Dayanand Kumbhar > > > > > > > > > > > > > > > > > > > > > > > > > >