Re: wget or curl and yousendit.com
On 10/14/2012 12:03 AM, JD wrote:
On 10/13/2012 06:26 PM, Mark LaPierre wrote:
Here's a bit of a guide to build your own script.
# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE";
sleep 1
# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc";
"rundown.php?prgId=2" | sort | uniq | awk -F"\"" '{print $4}' | awk
-F"?" '{print $1}'`
do
wget $URL 2> /dev/null &
sleep 1
done
Thank you Mark.
After looking into the source page containing all the links, and saw how
the
URL of each file to download is constructed, I built and ran the
following script
which worked like a charm.
PS: The first URL given to wget came to me in an email message sent to
me by
yousendit.com because the uploader has me in his distribution list on
yousendit.com.
#!/bin/bash
wget -c -O ysi.txt
'https://www.yousendit.com/dl?phi_action=app/orchestrateDownload&rurl=httpsetc'
grep 'directDownload?' ysi.txt | sed -e
's/.*directDownload?/https:\/\/www.yousendit.com\/directDownload?/' -e
's/=bas.*$/=bas/' | sort | uniq > urls # so that I can examine this file
in case my sed script has an error
while read url; do
wget -c -ndH "$url"
done < urls
Thanx for the inspiration Mark.
Great minds think alike. ;-)
--
_
°v°
/(_)\
^ ^ Mark LaPierre
Registerd Linux user No #267004
https://linuxcounter.net/
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
Re: wget or curl and yousendit.com
On 10/13/2012 06:26 PM, Mark LaPierre wrote:
Here's a bit of a guide to build your own script.
# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE";
sleep 1
# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc";
"rundown.php?prgId=2" | sort | uniq | awk -F"\"" '{print $4}' | awk
-F"?" '{print $1}'`
do
wget $URL 2> /dev/null &
sleep 1
done
Thank you Mark.
After looking into the source page containing all the links, and saw how the
URL of each file to download is constructed, I built and ran the
following script
which worked like a charm.
PS: The first URL given to wget came to me in an email message sent to me by
yousendit.com because the uploader has me in his distribution list on
yousendit.com.
#!/bin/bash
wget -c -O ysi.txt
'https://www.yousendit.com/dl?phi_action=app/orchestrateDownload&rurl=httpsetc'
grep 'directDownload?' ysi.txt | sed -e
's/.*directDownload?/https:\/\/www.yousendit.com\/directDownload?/' -e
's/=bas.*$/=bas/' | sort | uniq > urls # so that I can examine this
file in case my sed script has an error
while read url; do
wget -c -ndH "$url"
done < urls
Thanx for the inspiration Mark.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
Re: wget or curl and yousendit.com
On 10/13/2012 06:26 PM, Mark LaPierre wrote:
On 10/13/2012 08:12 PM, Ian Malone wrote:
On 13 October 2012 23:08, JD wrote:
Is there a way to use wget or curl or clive to download files from
yousendit.com?
Would appreciate some examples.
Each file link, when you cright-click it, takes you to the full link
of the
file,
and you can then use that link as an arg to wget.
If feeling brave you could script link extraction from the page and a
second curl request, alternatively wget's recursive -r option may be
what you want:
http://www.linuxforums.org/forum/applications/145133-get-complete-webpage-wget-curl.html
Here's a bit of a guide to build your own script.
# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE";
sleep 1
# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc";
"rundown.php?prgId=2" | sort | uniq | awk -F"\"" '{print $4}' | awk
-F"?" '{print $1}'`
do
wget $URL 2> /dev/null &
sleep 1
done
Will try it. Thanks for the example.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
Re: wget or curl and yousendit.com
On 10/13/2012 08:12 PM, Ian Malone wrote:
On 13 October 2012 23:08, JD wrote:
Is there a way to use wget or curl or clive to download files from
yousendit.com?
Would appreciate some examples.
Each file link, when you cright-click it, takes you to the full link of the
file,
and you can then use that link as an arg to wget.
If feeling brave you could script link extraction from the page and a
second curl request, alternatively wget's recursive -r option may be
what you want:
http://www.linuxforums.org/forum/applications/145133-get-complete-webpage-wget-curl.html
Here's a bit of a guide to build your own script.
# GET THE WEB PAGE
wget -q -O "rundown.php?prgId=2"
"http://www.npr.org/templates/rundowns/rundown.php?prgId=2&prgDate=$DATE";
sleep 1
# NOW STRIP OUT THE URLs AND GET THE MP3 FILES
for URL in `grep "http://pd.npr.org/anon.npr-mp3/npr/atc";
"rundown.php?prgId=2" | sort | uniq | awk -F"\"" '{print $4}' | awk
-F"?" '{print $1}'`
do
wget $URL 2> /dev/null &
sleep 1
done
--
_
°v°
/(_)\
^ ^ Mark LaPierre
Registerd Linux user No #267004
https://linuxcounter.net/
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
Re: wget or curl and yousendit.com
On 13 October 2012 23:08, JD wrote: > >>> >>> Is there a way to use wget or curl or clive to download files from >>> yousendit.com? >>> Would appreciate some examples. > Each file link, when you cright-click it, takes you to the full link of the > file, > and you can then use that link as an arg to wget. > > > > If feeling brave you could script link extraction from the page and a second curl request, alternatively wget's recursive -r option may be what you want: http://www.linuxforums.org/forum/applications/145133-get-complete-webpage-wget-curl.html -- imalone http://ibmalone.blogspot.co.uk -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: wget or curl and yousendit.com
On 10/12/2012 11:47 PM, Mark LaPierre wrote: On 10/12/2012 09:50 PM, JD wrote: Is there a way to use wget or curl or clive to download files from yousendit.com? Would appreciate some examples. It looks like you have to log in to yousendit to get access to your file. That would indicate that it is not scriptable with anything like wget. You would have to get the login page and then provide the required credentials. There is almost always a Captcha that you have to read and decode put there just to prevent the kind of thing you want to do. Actually, no. I am not trying to access my own files that are UPLOADED to yousendit.com I am trying to download files that another user uplpoaded and made available to a distribution list. Usually, I have to click on each link and download it - no login required. But in some of the email notifications I get from yousendit.com, the message contains many files to download, and I am trying to avoid having to click on each and every one, to download them. Each file link, when you cright-click it, takes you to the full link of the file, and you can then use that link as an arg to wget. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: wget or curl and yousendit.com
On 10/12/2012 09:50 PM, JD wrote: Is there a way to use wget or curl or clive to download files from yousendit.com? Would appreciate some examples. It looks like you have to log in to yousendit to get access to your file. That would indicate that it is not scriptable with anything like wget. You would have to get the login page and then provide the required credentials. There is almost always a Captcha that you have to read and decode put there just to prevent the kind of thing you want to do. -- _ °v° /(_)\ ^ ^ Mark LaPierre Registerd Linux user No #267004 https://linuxcounter.net/ -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
wget or curl and yousendit.com
Is there a way to use wget or curl or clive to download files from yousendit.com? Would appreciate some examples. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
