Re: [blfs-dev] Proposal of patch to correct bad URL
Hi
Happy new year, I wish you health, happiness and success in your projects.
Le 11/12/2013 21:17, Bruce Dubbs a écrit :
That will be useful. Thanks.
What might also be helpful is to create a script to extract those links
from the book and automate the check.
-- Bruce
I send you a patch to automate the checking of all links contained in
the book.
I add the test-all-links fonction in the Makefile (make
test-all-links). This fonction call a xls file
(stylesheets/list_url.xsl) to extract a list of URLs from the book.
This fonction don't check the package links (use make test-links to
check it)
You can find a list of bad urls in the file $(BASEDIR)/true_bad_urls,
and the list of xml files using this urls in the file
$(BASEDIR)/broken_urls.
I hope this can help you
When I study stylesheets to write this script, I saw that it is
possible to generate a epub3 file of the LFS book. It is planned for
the next release of LFS to have a epub3 file of the LFS book ??
Regards,
Denis
Index: Makefile
===
--- Makefile (révision 12508)
+++ Makefile (copie de travail)
@@ -160,6 +160,38 @@
fi; \
done
+test-all-links: $(BASEDIR)/test-all-links
+$(BASEDIR)/test-all-links: $(RENDERTMP)/blfs-full.xml
+ @echo Generating test-all-links file...
+ $(Q)mkdir -p $(BASEDIR)
+ $(Q)xsltproc --nonet \
+ --output $(BASEDIR)/test-all-links stylesheets/list_url.xsl \
+ $(RENDERTMP)/blfs-full.xml
+
+ @echo Checking URLs, first pass...
+ $(Q)rm -f $(BASEDIR)/{good,bad,true_bad,broken}_urls
+ $(Q)for URL in `uniq $(BASEDIR)/test-all-links`; do \
+ wget --spider --tries=2 --timeout=60 $$URL /dev/null 21; \
+ if test $$? -ne 0 ; then echo $$URL $(BASEDIR)/bad_urls ; \
+ else echo $$URL $(BASEDIR)/good_urls 21; \
+ fi; \
+ done
+
+ @echo Checking URLs, second pass...
+ $(Q)for URL2 in `cat $(BASEDIR)/bad_urls`; do \
+ wget --spider --tries=2 --timeout=60 $$URL2 /dev/null 21; \
+ if test $$? -ne 0 ; then echo $$URL2 $(BASEDIR)/true_bad_urls ; \
+ else echo $$URL2 $(BASEDIR)/good_urls 21; \
+ fi; \
+ done
+
+ @echo Generating broken_urls file ...
+ $(Q)for URL in `cat $(BASEDIR)/true_bad_urls`; do \
+ grep -r $$URL * $(BASEDIR)/broken_urls ; \
+ done
+
+ $(Q)sed -i $(BASEDIR)/broken_urls -e /^tmp/d -e /xml~/d
+
dump-commands: $(DUMPDIR)
$(DUMPDIR): $(RENDERTMP)/blfs-full.xml
@echo Dumping book commands...
Index: stylesheets/list_url.xsl
===
--- stylesheets/list_url.xsl (révision 0)
+++ stylesheets/list_url.xsl (copie de travail)
@@ -0,0 +1,26 @@
+?xml version='1.0' encoding='ISO-8859-1'?
+
+!-- Create a list of broken URLs listed in dependencies. --
+
+xsl:stylesheet xmlns:xsl=http://www.w3.org/1999/XSL/Transform;
+version=1.0
+
+ xsl:output method=text/
+
+ xsl:template match=/
+ xsl:apply-templates select=//ulink mode=full
+ xsl:sort select=@url order=ascending/
+ /xsl:apply-templates
+ /xsl:template
+
+ xsl:template match=ulink mode=full
+!-- To check only url in text and not check url of packages (you can check it with test-links in Makefile --
+xsl:if test=not(ancestor::itemizedlist)
+xsl:if test=not(contains(@url, 'file:') or contains(@url, 'wiki.linuxfromscratch.org') or contains(@url, 'mailto') or contains(@url, 'localhost')) and (contains(@url, 'http:') or contains(@url, 'ftp:') or contains(@url, 'https:'))
+ xsl:value-of select=@url/
+ xsl:text#x0a;/xsl:text
+/xsl:if
+/xsl:if
+ /xsl:template
+
+/xsl:stylesheet
--
http://linuxfromscratch.org/mailman/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
On 12/12/2013 09:30 PM, Denis Mugnier wrote: I send another patch to fix all the broken links found. When I don't find a correct URL, I comment the line. Thanks, fixed at r12385. -- Igor Živković http://www.slashtime.net/ -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Le 11/12/2013 21:17, Bruce Dubbs a écrit : I check and fixe URLs of external dependencies. Tomorrow I will send you a patch to fix other broken links. That will be useful. Thanks. What might also be helpful is to create a script to extract those links from the book and automate the check. -- Bruce I send another patch to fix all the broken links found. When I don't find a correct URL, I comment the line. I can create a script to extract all the link of the BLFS book and checking. Regards, Denis Index: BOOK/general/prog/other-tools.xml === --- BOOK/general/prog/other-tools.xml (révision 12379) +++ BOOK/general/prog/other-tools.xml (copie de travail) @@ -996,10 +996,12 @@ paraProject Home Page: ulink url=http://lavape.sourceforge.net/index.htm//para /listitem -listitem +!-- URL broken + listitem paraProject Home Page: ulink url=http://javalab.cs.uni-bonn.de/research/darwin/#The%20Lava%20Language//para /listitem +-- listitem paraProject Home Page: ulink url=http://mathias.tripod.com/IavaHomepage.html//para @@ -1024,11 +1026,11 @@ itemizedlist spacing=compact listitem paraProject Home Page: ulink - url=http://www.mercury.csse.unimelb.edu.au///para + url=http://mercurylang.org///para /listitem listitem paraDownload Location: ulink - url=http://www.mercury.csse.unimelb.edu.au/download.html//para + url=http://mercurylang.org/download.html//para /listitem /itemizedlist @@ -1377,7 +1379,7 @@ /listitem listitem paraDownload Location: ulink - url=ftp://ftp.cs.rit.edu/pub/mirrors/SmartEiffel///para + url=https://gforge.inria.fr/frs/?group_id=184//para /listitem /itemizedlist Index: BOOK/general/prog/php.xml === --- BOOK/general/prog/php.xml (révision 12379) +++ BOOK/general/prog/php.xml (copie de travail) @@ -99,7 +99,7 @@ xref linkend=x-window-system/, ulink url=http://www.fastio.com/;ClibPDF/ulink, ulink url=https://bitbucket.org/libgd/gd-libgd/downloads;GD/ulink, -ulink url=http://www.sfr-fresh.com/linux/misc/t1lib;t1lib/ulink, and +ulink url=http://fossies.org/linux/misc/t1lib-5.1.2.tar.gz/;t1lib/ulink, and ulink url=http://partners.adobe.com/asn/acrobat/forms.jsp;FDF Toolkit/ulink/para bridgehead renderas=sect4Optional Web Utilities/bridgehead @@ -109,7 +109,7 @@ ulink url=http://www.hyperwave.com/;Hyperwave/ulink, ulink url=http://www.roxen.com/products/webserver/;Roxen WebServer/ulink, ulink url=http://caudium.net/;Caudium/ulink, and -ulink url=http://www.openwddx.org/;WDDX/ulink/para +ulink url=https://wddxnet.codeplex.com/;WDDX/ulink/para bridgehead renderas=sect4Optional Data Management Utilities and Libraries/bridgehead para role=optionalxref linkend=openldap/, Index: BOOK/xfce/apps/xfce4-mixer.xml === --- BOOK/xfce/apps/xfce4-mixer.xml (révision 12379) +++ BOOK/xfce/apps/xfce4-mixer.xml (copie de travail) @@ -81,10 +81,12 @@ xref linkend=xfce4-panel/ /para +!-- URL broken bridgehead renderas=sect4Optional/bridgehead -para role=optional + para role=optional ulink url=http://kaizer.se/wiki/keybinder/;Keybinder/ulink /para +-- para condition=html role=usernotes User Notes: ulink url=blfs-wiki;/xfce4-mixer/ Index: BOOK/networking/netlibs/libnice.xml === --- BOOK/networking/netlibs/libnice.xml (révision 12379) +++ BOOK/networking/netlibs/libnice.xml (copie de travail) @@ -90,7 +90,7 @@ para role=optional xref linkend=gst-plugins-base/, xref linkend=gtk-doc/, and - ulink url=http://gupnp.org/;gupnp-igd/ulink + ulink url=https://wiki.gnome.org/action/show/Projects/GUPnP;gupnp-igd/ulink /para para condition=html role=usernotesUser Notes: Index: BOOK/multimedia/videoutils/ffmpeg.xml === --- BOOK/multimedia/videoutils/ffmpeg.xml (révision 12379) +++ BOOK/multimedia/videoutils/ffmpeg.xml (copie de travail) @@ -130,10 +130,10 @@ ulink url=https://ieee1394.wiki.kernel.org/index.php/Libraries;libiec61883/ulink, ulink url=https://github.com/dekkers/libilbc;libilbc/ulink, ulink url=http://sourceforge.net/projects/modplug-xmms/;libmodplug/ulink, - ulink url=http://svn.mplayerhq.hu/nut/src/trunk/;libnut/ulink (SVN checkout), + ulink url=https://github.com/Distrotech/libnut;libnut/ulink (Git checkout), ulink
Re: [blfs-dev] Proposal of patch to correct bad URL
On 2013-12-10 21:15, Denis Mugnier wrote: Hi, For the french translation of BLFS, I check the URLs in the book. I found some bad URL, so I propose a patch to correct it. It is a first patch, another come when I have time ;o) Thanks, fixed at r12370. Could you send the next one as an attachment so it doesn't get wrapped by the mail client, please? -- Igor Živković http://www.slashtime.net/ -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
On 12/11/2013 3:47 AM, Igor Živković wrote: On 2013-12-10 21:15, Denis Mugnier wrote: Hi, For the french translation of BLFS, I check the URLs in the book. I found some bad URL, so I propose a patch to correct it. It is a first patch, another come when I have time ;o) Thanks, fixed at r12370. Could you send the next one as an attachment so it doesn't get wrapped by the mail client, please? IIRC, there is already an entry in the Makefile of the root BLFS tree that will check the validity of all the URLs. -- Randy -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
akhiezer wrote: Do you automate the testing of the book's urls for brokenness: would have caught the (apparent) errors in the patch; and save a lot of manual work in the medium-/long- term. Not sure if Bruce's test-for-updates cron script would incorporate test-for-brokenness, or if you have access to that. The scripts work from the generated wget-list. There are some errors with the scripts as I haven't updated them in about 6 weeks. I need to do that. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Le 11/12/2013 17:17, Randy McMurchy a écrit : IIRC, there is already an entry in the Makefile of the root BLFS tree that will check the validity of all the URLs. I try the testing-url entry of the Makefile, but, I think that not all the URLs are tested. Anyone can confirm that ? -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Denis Mugnier wrote: Le 11/12/2013 17:17, Randy McMurchy a écrit : IIRC, there is already an entry in the Makefile of the root BLFS tree that will check the validity of all the URLs. I try the testing-url entry of the Makefile, but, I think that not all the URLs are tested. Anyone can confirm that ? I think you mean the test-links target. That checks the urls in the wget-list. The wget-list is generated from the xml and does not include patches or other urls mentioned in the book (e.g. external dependencies). -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Le 11/12/2013 20:59, Bruce Dubbs a écrit : I think you mean the test-links target. That checks the urls in the wget-list. The wget-list is generated from the xml and does not include patches or other urls mentioned in the book (e.g. external dependencies). -- Bruce I check and fixe URLs of external dependencies. Tomorrow I will send you a patch to fix other broken links. Denis -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Denis Mugnier wrote: Le 11/12/2013 20:59, Bruce Dubbs a écrit : I think you mean the test-links target. That checks the urls in the wget-list. The wget-list is generated from the xml and does not include patches or other urls mentioned in the book (e.g. external dependencies). I check and fixe URLs of external dependencies. Tomorrow I will send you a patch to fix other broken links. That will be useful. Thanks. What might also be helpful is to create a script to extract those links from the book and automate the check. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
[blfs-dev] Proposal of patch to correct bad URL
Hi, For the french translation of BLFS, I check the URLs in the book. I found some bad URL, so I propose a patch to correct it. It is a first patch, another come when I have time ;o) thanks for the project. Best Regard Denis French translation team Index: BOOK/general/genutils/graphviz.xml === --- BOOK/general/genutils/graphviz.xml (révision 12363) +++ BOOK/general/genutils/graphviz.xml (copie de travail) @@ -109,7 +109,7 @@ (applicationSWIG/application must be installed or no bindings will be built), xref linkend=guile/, xref linkend=openjdk/, - ulink url=http://www.iolanguage.com/about/;Io/ulink, + ulink url=http://www.iolanguage.org/;Io/ulink, xref linkend=lua/, ulink url=http://www.mono-project.com/Main_Page;Mono/ulink, ulink url=http://ocaml.org/;OCaml/ulink, Index: BOOK/general/genutils/imagemagick.xml === --- BOOK/general/genutils/imagemagick.xml (révision 12363) +++ BOOK/general/genutils/imagemagick.xml (copie de travail) @@ -171,7 +171,7 @@ para role=optional ulink url=http://www.blender.org/;Blender/ulink, ulink url=http://dejavu.sourceforge.net/;DejaVu fonts/ulink, - ulink url=http://www.artifex.com/downloads/;GhostPCL/ulink or + ulink url=http://www.ghostscript.com/download/gpcldnld.html/;GhostPCL/ulink or ulink url=http://www.ghostscript.com/GhostPCL.html;GhostPDL/ulink, xref linkend=gs/, xref linkend=gimp/, Index: BOOK/general/sysutils/autofs.xml === --- BOOK/general/sysutils/autofs.xml(révision 12363) +++ BOOK/general/sysutils/autofs.xml(copie de travail) @@ -229,7 +229,7 @@ feature is not configured in these instructions. For additional configuration information, see the man pages for auto.master(5). There are also web resources such as this ulink - url='http://en.gentoo-wiki.com/wiki/Autofs'AUTOFS + url='http://wiki.gentoo.org/wiki/Autofs'AUTOFS HOWTO/ulink available./para /sect3 Index: BOOK/general/genlib/gmime.xml === --- BOOK/general/genlib/gmime.xml (révision 12363) +++ BOOK/general/genlib/gmime.xml (copie de travail) @@ -94,7 +94,7 @@ xref linkend=docbook-utils/, xref linkend=gpgme/, xref linkend=gtk-doc/ and - ulink url=http://go-mono.com/sources/;Gtk#/ulink (requires + ulink url=http://http://download.mono-project.com/sources/;Gtk#/ulink (requires ulink url=http://www.mono-project.com/Main_Page;Mono/ulink) /para Index: BOOK/postlfs/config/users.xml === --- BOOK/postlfs/config/users.xml (révision 12363) +++ BOOK/postlfs/config/users.xml (copie de travail) @@ -48,7 +48,7 @@ assigned will always be above these cutoff values./para paraAdditionally, the ulink - url='http://refspecs.freestandards.org/LSB_3.0.0/LSB-Core-generic/LSB-Core-generic/usernames.html' + url='http://http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/usernames.html' Linux Standard Base/ulink recommends that system uid and gid values should be below 100./para Index: BOOK/postlfs/security/tripwire.xml === --- BOOK/postlfs/security/tripwire.xml (révision 12363) +++ BOOK/postlfs/security/tripwire.xml (copie de travail) @@ -220,9 +220,10 @@ about, remove the 'x' before the filename in the report and type command:wq/command./para + !-- 10-12-2013 bad URL and no good URL found paraA good summary of tripwire operations can be found at ulink url=http://va-holladays.no-ip.info:2200/tools/security-docs/tripwire-v1.0.pdf/./para - + -- /sect3 sect3 Index: BOOK/postlfs/security/sudo.xml === --- BOOK/postlfs/security/sudo.xml (révision 12363) +++ BOOK/postlfs/security/sudo.xml (copie de travail) @@ -84,7 +84,7 @@ an xref linkend=server-mail/ (that provides a commandsendmail/command command), xref linkend=openldap/, - ulink url=ftp://ftp.nrl.navy.mil/pub/security/opie;Opie/ulink and + ulink url=http://sourceforge.net/projects/opie/files/;Opie/ulink and ulink url=http://www.rsa.com/node.aspx?id=1156;SecurID/ulink /para Index: BOOK/postlfs/security/linux-pam.xml === --- BOOK/postlfs/security/linux-pam.xml (révision 12363) +++ BOOK/postlfs/security/linux-pam.xml (copie de travail) @@ -242,7 +242,7 @@ The applicationPAM/application man page (commandman pam/command) provides a
Re: [blfs-dev] Proposal of patch to correct bad URL
Date: Tue, 10 Dec 2013 21:15:41 +0100 From: Denis Mugnier myo...@orange.fr To: blfs Dev ml blfs-dev@linuxfromscratch.org Subject: [blfs-dev] Proposal of patch to correct bad URL Hi, For the french translation of BLFS, I check the URLs in the book. I found some bad URL, so I propose a patch to correct it. . . xref linkend=gtk-doc/ and - ulink url=http://go-mono.com/sources/;Gtk#/ulink (requires + ulink url=http://http://download.mono-project.com/sources/;Gtk#/ulink (requires - double-protocol in url. . . - url='http://refspecs.freestandards.org/LSB_3.0.0/LSB-Core-generic/LSB-Core-generic/usernames.html' + url='http://http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/usernames.html' Linux Standard Base/ulink recommends that system uid and gid - double-protocol in url. rgds, akiezer -- -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Date: Tue, 10 Dec 2013 21:21:19 + From: lf...@cruziero.com (akhiezer) To: BLFS Development List blfs-dev@linuxfromscratch.org Subject: Re: [blfs-dev] Proposal of patch to correct bad URL Date: Tue, 10 Dec 2013 21:15:41 +0100 From: Denis Mugnier myo...@orange.fr To: blfs Dev ml blfs-dev@linuxfromscratch.org Subject: [blfs-dev] Proposal of patch to correct bad URL Hi, For the french translation of BLFS, I check the URLs in the book. I found some bad URL, so I propose a patch to correct it. . . xref linkend=gtk-doc/ and - ulink url=http://go-mono.com/sources/;Gtk#/ulink (requires + ulink url=http://http://download.mono-project.com/sources/;Gtk#/ulink (requires - double-protocol in url. . . - url='http://refspecs.freestandards.org/LSB_3.0.0/LSB-Core-generic/LSB-Core-generic/usernames.html' + url='http://http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/usernames.html' Linux Standard Base/ulink recommends that system uid and gid - double-protocol in url. Do you automate the testing of the book's urls for brokenness: would have caught the (apparent) errors in the patch; and save a lot of manual work in the medium-/long- term. Not sure if Bruce's test-for-updates cron script would incorporate test-for-brokenness, or if you have access to that. rgds, akiezer -- -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page -- -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
Re: [blfs-dev] Proposal of patch to correct bad URL
Le 10/12/2013 23:30, akhiezer a écrit : Do you automate the testing of the book's urls for brokenness: would have caught the (apparent) errors in the patch; and save a lot of manual work in the medium-/long- term. Not sure if Bruce's test-for-updates cron script would incorporate test-for-brokenness, or if you have access to that. Yes, I automate the testing of URLs. My first goal is not to test the urls, but to compare the URLs between the English and French translation of BLFS. Together, I check the URLs with a wget --spider command. Currently, my script don't work properly, but when I have debug it, I can post here the testing of book's urls part. Rgds Denis -- http://linuxfromscratch.org/mailman/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
