watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140424080424.gg11...@an3as.eu
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On Thu, Apr 24, 2014 at 4:04 PM, Andreas Tille wrote:
Any idea how we could make uscan capable to detect and download the latest
source anyway?
Use a match based on the zip download and downloadurlmangle to
transform that to a src tarball.
a href=/download/mkgmap-r3226.zip data-bind=attr: {href:
current().full_name}
--
bye,
pabs
http://wiki.debian.org/PaulWise
--
To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
https://lists.debian.org/CAKTje6FtVDHCgYCSSt1T7OWdUh=7zsdqabh70owexy_jyh0...@mail.gmail.com
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E signature.asc Description: OpenPGP digital signature
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 09:44, Daniel Lintott wrote: On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, Having just tried this again.. it only half works... Output from uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/ http://www.mkgmap.org.uk/download/ mkgmap-r([\d]+)\.zip -- Found the following matching hrefs: /download/mkgmap-r3226.zip (3226) Newest version on remote site is 3226, local version is 0.0.0+svn2981 (mangled local version number 2981) = Newer version available from http://www.mkgmap.org.uk/download/mkgmap-r3226-src.tar.gz -- Downloading updated package mkgmap-r3226.zip -- Scan finished So it seems as if uscan is ignoring the downloadurlmangle when it comes to actually downloading the file! Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E signature.asc Description: OpenPGP digital signature
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 10:32, Daniel Lintott wrote: On 24/04/14 09:44, Daniel Lintott wrote: On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, Having just tried this again.. it only half works... Output from uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/ http://www.mkgmap.org.uk/download/ mkgmap-r([\d]+)\.zip -- Found the following matching hrefs: /download/mkgmap-r3226.zip (3226) Newest version on remote site is 3226, local version is 0.0.0+svn2981 (mangled local version number 2981) = Newer version available from http://www.mkgmap.org.uk/download/mkgmap-r3226-src.tar.gz -- Downloading updated package mkgmap-r3226.zip -- Scan finished So it seems as if uscan is ignoring the downloadurlmangle when it comes to actually downloading the file! Regards, After digging into even more... hate to leave a problem unsolved... I think I cracked it! In addition to the the downloadurlmangle, a filenamemangle is needed to save the downloaded file with the correct name. filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ Which removes the download directory and names the file correctly as a tar.gz Hope that helps... Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E signature.asc Description: OpenPGP digital signature
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
Hi Daniel, thanks for your continuous trials to help me. :-) On Thu, Apr 24, 2014 at 11:22:22AM +0100, Daniel Lintott wrote: On 24/04/14 10:32, Daniel Lintott wrote: On 24/04/14 09:44, Daniel Lintott wrote: On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, Having just tried this again.. it only half works... Output from uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/ http://www.mkgmap.org.uk/download/ mkgmap-r([\d]+)\.zip -- Found the following matching hrefs: /download/mkgmap-r3226.zip (3226) Newest version on remote site is 3226, local version is 0.0.0+svn2981 (mangled local version number 2981) = Newer version available from http://www.mkgmap.org.uk/download/mkgmap-r3226-src.tar.gz -- Downloading updated package mkgmap-r3226.zip -- Scan finished So it seems as if uscan is ignoring the downloadurlmangle when it comes to actually downloading the file! Regards, After digging into even more... hate to leave a problem unsolved... I think I cracked it! In addition to the the downloadurlmangle, a filenamemangle is needed to save the downloaded file with the correct name. filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ Which removes the download directory and names the file correctly as a tar.gz Hope that helps... Not really since I can not confirm that I get anything else but: no matching hrefs for watch line I guess we somehow need to go with Paul's hint. All your hints are based on changing the regexp for the source download by droping the '-src' part from the file name. Since we really want to fetch the source and not simply the binary jar this is not helpful. However, I somehow need to make some sense out of Paul's hint which I did not fully understand admittedly. Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140424113214.gi11...@an3as.eu
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 12:32, Andreas Tille wrote: Hi Daniel, thanks for your continuous trials to help me. :-) No problem! I enjoy a good bit of problem solving! On Thu, Apr 24, 2014 at 11:22:22AM +0100, Daniel Lintott wrote: Hope that helps... Not really since I can not confirm that I get anything else but: no matching hrefs for watch line I guess we somehow need to go with Paul's hint. All your hints are based on changing the regexp for the source download by droping the '-src' part from the file name. Since we really want to fetch the source and not simply the binary jar this is not helpful. However, I somehow need to make some sense out of Paul's hint which I did not fully understand admittedly. What Paul was hinting at was using using the downloadurlmangle as per my first email. What happens here is we get the href for the binary download and then mangle it to get the source tarball, by replacing .zip with -src.tar.gz s/\.zip/-src.tar.gz/ Next we have to make uscan save the file it downloads with the correct filename... filenamemangle. filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ Here we replace /download/mkgmap-r.zip with mkgmap-r-src.tar.gz to match the name of the that is actually downloaded. I have also attached a copy of the watch file I used to test the mangles. Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E version=3 opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/,filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ \ http://www.mkgmap.org.uk/download/mkgmap-r([\d]+).zip signature.asc Description: OpenPGP digital signature
