Re: Using xmllint to test our LyXHTML export
Am Samstag, 28. Mai 2016 um 19:43:27, schrieb Kornel Benko > Am Samstag, 28. Mai 2016 um 17:36:14, schrieb Kornel Benko > ... > > > > I found the cause in layouts. With the attached patch for the verbatim and > > code insets are making some tests happy. > > > > > Next error in attached example. As soon as we dissolve the 'include > TestSummary.lyx' from branch, > the document is exportable. > Also removing the text 'Small text' in TestSummary.lyx makes it exportable. > > Next one found: Minimal example created from es/UserGuide.lyx Kornel TestUserGuide.lyx Description: application/lyx signature.asc Description: This is a digitally signed message part.
Re: Using xmllint to test our LyXHTML export
Am Samstag, 28. Mai 2016 um 17:36:14, schrieb Kornel Benko ... > > I found the cause in layouts. With the attached patch for the verbatim and > code insets are making some tests happy. > Next error in attached example. As soon as we dissolve the 'include TestSummary.lyx' from branch, the document is exportable. Also removing the text 'Small text' in TestSummary.lyx makes it exportable. Kornel TestSummary.lyx Description: application/lyx TestBranchThesis.lyx Description: application/lyx signature.asc Description: This is a digitally signed message part.
Re: Using xmllint to test our LyXHTML export
Am Donnerstag, 26. Mai 2016 um 17:16:30, schrieb Scott Kostyshak > On Thu, May 26, 2016 at 06:44:39PM +0200, Kornel Benko wrote: > > Am Mittwoch, 25. Mai 2016 um 16:48:58, schrieb Kornel Benko > > > Am Montag, 23. Mai 2016 um 18:03:27, schrieb Scott Kostyshak > > > > > > > On Mon, May 23, 2016 at 03:55:59PM +, Guenter Milde wrote: > > > > > On 2016-05-23, Kornel Benko wrote: > > > > > > Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde > > > > > > > > > > > >> On 2016-05-22, Scott Kostyshak wrote: > > > > > > > > > > >> > Uwe and Georg have recently found and fixed a bug in our > > > > > >> > internal XHTML > > > > > >> > export. Why not test the export automatically? > > > > > > > > > > ... > > > > > > > > > > > I have a patch ready. > > > > > > > > > > Fine, thanks. > > > > > > > > > > > But now many of xhtml export tests are failing. > > > > > > > > > > This should not deter us from proper testing, should it? > > > > > > > > In my opinion, the important thing is to make sure no new tests go from > > > > passsing to failinig. It is possible to easily catch future regressions > > > > with the tests. I agree with you (below) that inverting a lot of tests > > > > makes sense. This way, we can (1) catch new regressions and (2) if > > > > someone does come along and wants to give some love to the HTML export > > > > we can easily provide them with some places to start (in addition to the > > > > trac bugs we have). > > > > > > > > Scott > > > > > > > > > > > > > > Do you have a list of failing tests? > > > > > > Yes, I have. But the list is huge. > > > 48% tests passed, 186 tests failed out of 356 > > > > > > > > Did you check for the failure reasons? > > > > > > Some of them are really bad, like in doc/MergedManuals.xhtml:9270 > > > < class="verbatim"> > > class="verbatim_item">This is Verbatim. > > > > > > others are OK in firefox. > > > > > > > > I may be we have to invert a lot of tests until the HTML export is > > > > > fixed. > > > > > Maybe we can also introduce a flag for "half-OK" (not passing the > > > > > xmllint test > > > > > but opening OK in a browser). > > > > > > > > > > No, for this the number of failures is too big IMHO. > > > > Hm, using xmllint with the '--sax' parameter shows no errors. Still > > MergedManuals.xhtml is erroneous (e.g. not displayable in firefox). > > Checking the includes, the error is in UserGuide.lyx. > > It turns out that our xhtml cannot handle verbatim environment. > > > > Attached a minimal example: > > > > Richard could you have a look at it? > > Josh (now CC'ed), I remember that at some point you made some fixes to > our XHTML export. Are you by chance interested in working on it further? > We are considering implementing tests for our XHTML export but they are > only useful if we have someone who might be interested in fixing bugs > that our tests find. No pressure at all, just curious if that would be > fun for you. > > Scott I found the cause in layouts. With the attached patch for the verbatim and code insets are making some tests happy. Korneldiff --git a/lib/layouts/jss.layout b/lib/layouts/jss.layout index 47e3dae..1055043 100644 --- a/lib/layouts/jss.layout +++ b/lib/layouts/jss.layout @@ -226,7 +226,7 @@ Style "Code" Series Bold ColorGreen EndFont - HTMLTag + HTMLTagpre HTMLItem p End diff --git a/lib/layouts/stdlayouts.inc b/lib/layouts/stdlayouts.inc index 8be0d24..dbde134 100644 --- a/lib/layouts/stdlayouts.inc +++ b/lib/layouts/stdlayouts.inc @@ -91,7 +91,7 @@ Style Verbatim Font Family Typewriter EndFont - HTMLTag + HTMLTag pre HTMLItem p End signature.asc Description: This is a digitally signed message part.
Re: Using xmllint to test our LyXHTML export
On Thu, May 26, 2016 at 06:44:39PM +0200, Kornel Benko wrote: > Am Mittwoch, 25. Mai 2016 um 16:48:58, schrieb Kornel Benko > > Am Montag, 23. Mai 2016 um 18:03:27, schrieb Scott Kostyshak > > > > > On Mon, May 23, 2016 at 03:55:59PM +, Guenter Milde wrote: > > > > On 2016-05-23, Kornel Benko wrote: > > > > > Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde > > > > > > > > > >> On 2016-05-22, Scott Kostyshak wrote: > > > > > > > > >> > Uwe and Georg have recently found and fixed a bug in our internal > > > > >> > XHTML > > > > >> > export. Why not test the export automatically? > > > > > > > > ... > > > > > > > > > I have a patch ready. > > > > > > > > Fine, thanks. > > > > > > > > > But now many of xhtml export tests are failing. > > > > > > > > This should not deter us from proper testing, should it? > > > > > > In my opinion, the important thing is to make sure no new tests go from > > > passsing to failinig. It is possible to easily catch future regressions > > > with the tests. I agree with you (below) that inverting a lot of tests > > > makes sense. This way, we can (1) catch new regressions and (2) if > > > someone does come along and wants to give some love to the HTML export > > > we can easily provide them with some places to start (in addition to the > > > trac bugs we have). > > > > > > Scott > > > > > > > > > > > Do you have a list of failing tests? > > > > Yes, I have. But the list is huge. > > 48% tests passed, 186 tests failed out of 356 > > > > > > Did you check for the failure reasons? > > > > Some of them are really bad, like in doc/MergedManuals.xhtml:9270 > > < class="verbatim"> > class="verbatim_item">This is Verbatim. > > > > others are OK in firefox. > > > > > > I may be we have to invert a lot of tests until the HTML export is > > > > fixed. > > > > Maybe we can also introduce a flag for "half-OK" (not passing the > > > > xmllint test > > > > but opening OK in a browser). > > > > > > > No, for this the number of failures is too big IMHO. > > Hm, using xmllint with the '--sax' parameter shows no errors. Still > MergedManuals.xhtml is erroneous (e.g. not displayable in firefox). > Checking the includes, the error is in UserGuide.lyx. > It turns out that our xhtml cannot handle verbatim environment. > > Attached a minimal example: > > Richard could you have a look at it? Josh (now CC'ed), I remember that at some point you made some fixes to our XHTML export. Are you by chance interested in working on it further? We are considering implementing tests for our XHTML export but they are only useful if we have someone who might be interested in fixing bugs that our tests find. No pressure at all, just curious if that would be fun for you. Scott signature.asc Description: PGP signature
Re: Using xmllint to test our LyXHTML export
Am Mittwoch, 25. Mai 2016 um 16:48:58, schrieb Kornel Benko > Am Montag, 23. Mai 2016 um 18:03:27, schrieb Scott Kostyshak > > > On Mon, May 23, 2016 at 03:55:59PM +, Guenter Milde wrote: > > > On 2016-05-23, Kornel Benko wrote: > > > > Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde > > > > > > > >> On 2016-05-22, Scott Kostyshak wrote: > > > > > > >> > Uwe and Georg have recently found and fixed a bug in our internal > > > >> > XHTML > > > >> > export. Why not test the export automatically? > > > > > > ... > > > > > > > I have a patch ready. > > > > > > Fine, thanks. > > > > > > > But now many of xhtml export tests are failing. > > > > > > This should not deter us from proper testing, should it? > > > > In my opinion, the important thing is to make sure no new tests go from > > passsing to failinig. It is possible to easily catch future regressions > > with the tests. I agree with you (below) that inverting a lot of tests > > makes sense. This way, we can (1) catch new regressions and (2) if > > someone does come along and wants to give some love to the HTML export > > we can easily provide them with some places to start (in addition to the > > trac bugs we have). > > > > Scott > > > > > > > > Do you have a list of failing tests? > > Yes, I have. But the list is huge. > 48% tests passed, 186 tests failed out of 356 > > > > Did you check for the failure reasons? > > Some of them are really bad, like in doc/MergedManuals.xhtml:9270 > < class="verbatim"> class="verbatim_item">This is Verbatim. > > others are OK in firefox. > > > > I may be we have to invert a lot of tests until the HTML export is fixed. > > > Maybe we can also introduce a flag for "half-OK" (not passing the xmllint > > > test > > > but opening OK in a browser). > > > > No, for this the number of failures is too big IMHO. Hm, using xmllint with the '--sax' parameter shows no errors. Still MergedManuals.xhtml is erroneous (e.g. not displayable in firefox). Checking the includes, the error is in UserGuide.lyx. It turns out that our xhtml cannot handle verbatim environment. Attached a minimal example: Richard could you have a look at it? Kornel testVerbatim.lyx Description: application/lyx signature.asc Description: This is a digitally signed message part.
Re: Using xmllint to test our LyXHTML export
Am Montag, 23. Mai 2016 um 18:03:27, schrieb Scott Kostyshak > On Mon, May 23, 2016 at 03:55:59PM +, Guenter Milde wrote: > > On 2016-05-23, Kornel Benko wrote: > > > Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde > > > > > >> On 2016-05-22, Scott Kostyshak wrote: > > > > >> > Uwe and Georg have recently found and fixed a bug in our internal XHTML > > >> > export. Why not test the export automatically? > > > > ... > > > > > I have a patch ready. > > > > Fine, thanks. > > > > > But now many of xhtml export tests are failing. > > > > This should not deter us from proper testing, should it? > > In my opinion, the important thing is to make sure no new tests go from > passsing to failinig. It is possible to easily catch future regressions > with the tests. I agree with you (below) that inverting a lot of tests > makes sense. This way, we can (1) catch new regressions and (2) if > someone does come along and wants to give some love to the HTML export > we can easily provide them with some places to start (in addition to the > trac bugs we have). > > Scott > > > > > Do you have a list of failing tests? Yes, I have. But the list is huge. 48% tests passed, 186 tests failed out of 356 > > Did you check for the failure reasons? Some of them are really bad, like in doc/MergedManuals.xhtml:9270 < class="verbatim">This is Verbatim. others are OK in firefox. > > I may be we have to invert a lot of tests until the HTML export is fixed. > > Maybe we can also introduce a flag for "half-OK" (not passing the xmllint > > test > > but opening OK in a browser). > No, for this the number of failures is too big IMHO. Kornel signature.asc Description: This is a digitally signed message part.
Re: Using xmllint to test our LyXHTML export
On Mon, May 23, 2016 at 03:55:59PM +, Guenter Milde wrote: > On 2016-05-23, Kornel Benko wrote: > > Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde > > > >> On 2016-05-22, Scott Kostyshak wrote: > > >> > Uwe and Georg have recently found and fixed a bug in our internal XHTML > >> > export. Why not test the export automatically? > > ... > > > I have a patch ready. > > Fine, thanks. > > > But now many of xhtml export tests are failing. > > This should not deter us from proper testing, should it? In my opinion, the important thing is to make sure no new tests go from passsing to failinig. It is possible to easily catch future regressions with the tests. I agree with you (below) that inverting a lot of tests makes sense. This way, we can (1) catch new regressions and (2) if someone does come along and wants to give some love to the HTML export we can easily provide them with some places to start (in addition to the trac bugs we have). Scott > > Do you have a list of failing tests? > > Did you check for the failure reasons? > > I may be we have to invert a lot of tests until the HTML export is fixed. > Maybe we can also introduce a flag for "half-OK" (not passing the xmllint test > but opening OK in a browser). signature.asc Description: PGP signature
Re: Using xmllint to test our LyXHTML export
On 2016-05-23, Kornel Benko wrote: > Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde > >> On 2016-05-22, Scott Kostyshak wrote: >> > Uwe and Georg have recently found and fixed a bug in our internal XHTML >> > export. Why not test the export automatically? ... > I have a patch ready. Fine, thanks. > But now many of xhtml export tests are failing. This should not deter us from proper testing, should it? Do you have a list of failing tests? Did you check for the failure reasons? I may be we have to invert a lot of tests until the HTML export is fixed. Maybe we can also introduce a flag for "half-OK" (not passing the xmllint test but opening OK in a browser). Günter
Re: Using xmllint to test our LyXHTML export
Am Montag, 23. Mai 2016 um 07:36:53, schrieb Guenter Milde
> On 2016-05-22, Scott Kostyshak wrote:
>
> > Uwe and Georg have recently found and fixed a bug in our internal XHTML
> > export. Why not test the export automatically?
>
> > These tests would be fast (compared to the LaTeX compilation tests we
> > have). I think our current XHTML tests just check that the exit code is
> > zero. We can extend these tests to work as follows:
>
> > 1. Export to .xhtml and check the exit code.
> > 2. Check the terminal output from LyX (I think we do this for other tests,
> > such
> > as the lyx2lyx tests).
> > 3. Use xmllint or something similar and check exit code.
>
> This would be a great improvement of the tests indeed.
>
> (I remember that there used to be more case of invalid xhtml and I am not
> sure whether we fixed everything...)
>
> > I know nothing about XHTML. Does anyone have experience with xmllint or
> > any other tool? The only other tool I saw mentioned when searching is
> > XMLStarlet. If someone does have experience, which tool do you
> > recommend? Also, I'd be curious which options you think should be used.
> > It seems to me that we would not want a very strict validation, and
> > basically want to check whether the file is expected to open in a
> > webbrowser without parser errors.
>
> We could start with a basic check and get more strict later...
>
>
> Günter
>
I have a patch ready. But now many of xhtml export tests are failing.
Kornel
signature.asc
Description: This is a digitally signed message part.
diff --git a/development/autotests/ExportTests.cmake b/development/autotests/ExportTests.cmake
index f448d80..562c639 100644
--- a/development/autotests/ExportTests.cmake
+++ b/development/autotests/ExportTests.cmake
@@ -27,6 +27,7 @@
#
find_package(Perl)
+find_program(XMLLINT_EXECUTABLE xmllint)
if(PERL_FOUND)
set(DVI_FORMATS "dvi" "dvi3")
@@ -401,6 +402,7 @@ foreach(libsubfolderx autotests/export lib/doc lib/examples lib/templates autote
-Dinverted=${inverted}
-DTOP_SRC_DIR=${TOP_SRC_DIR}
-DPERL_EXECUTABLE=${PERL_EXECUTABLE}
+-DXMLLINT_EXECUTABLE=${XMLLINT_EXECUTABLE}
-P "${TOP_SRC_DIR}/development/autotests/export.cmake")
setmarkedtestlabel(${TestName} ${mytestlabel}) # check for suspended pdf/dvi exports
endif()
diff --git a/development/autotests/export.cmake b/development/autotests/export.cmake
index 29487d4..4668ccd 100755
--- a/development/autotests/export.cmake
+++ b/development/autotests/export.cmake
@@ -25,6 +25,7 @@
# -Dinverted=[01] \
# -DTOP_SRC_DIR=${TOP_SRC_DIR}
# -DPERL_EXECUTABLE=${PERL_EXECUTABLE}
+# -DXMLLINT_EXECUTABLE=${XMLLINT_EXECUTABLE}
# -P "${TOP_SRC_DIR}/development/autotests/export.cmake"
#
@@ -138,6 +139,24 @@ else()
set(_err -1)
else()
message(STATUS "Expected result file \"${result_file_name}\" exists")
+ if (format MATCHES "xhtml")
+if (XMLLINT_EXECUTABLE)
+ # check the created xhtml file
+ execute_process(
+COMMAND ${XMLLINT_EXECUTABLE} --noout --html --valid "${result_file_name}"
+ERROR_VARIABLE xmlerr
+RESULT_VARIABLE _err)
+ if (NOT _err)
+# check if error output not empty
+if (NOT xmlerr STREQUAL "")
+ # Here, the check shows errors although the process
+ # exited with "0"
+ message(STATUS ${xmlerr})
+ set(_err -1)
+endif()
+ endif()
+endif()
+ endif()
endif()
endif()
endif()
@@ -147,7 +166,7 @@ if(inverted)
else()
string(COMPARE NOTEQUAL ${_err} 0 _erg)
endif()
-execute_process(COMMAND ${CMAKE_COMMAND} -E remove_directory "${TempDir}")
+#execute_process(COMMAND ${CMAKE_COMMAND} -E remove_directory "${TempDir}")
if(_erg)
message(STATUS "Exporting ${file}.lyx to ${format}")
message(FATAL_ERROR "Export failed")
Re: Using xmllint to test our LyXHTML export
On 2016-05-22, Scott Kostyshak wrote: > Uwe and Georg have recently found and fixed a bug in our internal XHTML > export. Why not test the export automatically? > These tests would be fast (compared to the LaTeX compilation tests we > have). I think our current XHTML tests just check that the exit code is > zero. We can extend these tests to work as follows: > 1. Export to .xhtml and check the exit code. > 2. Check the terminal output from LyX (I think we do this for other tests, > such > as the lyx2lyx tests). > 3. Use xmllint or something similar and check exit code. This would be a great improvement of the tests indeed. (I remember that there used to be more case of invalid xhtml and I am not sure whether we fixed everything...) > I know nothing about XHTML. Does anyone have experience with xmllint or > any other tool? The only other tool I saw mentioned when searching is > XMLStarlet. If someone does have experience, which tool do you > recommend? Also, I'd be curious which options you think should be used. > It seems to me that we would not want a very strict validation, and > basically want to check whether the file is expected to open in a > webbrowser without parser errors. We could start with a basic check and get more strict later... Günter
Using xmllint to test our LyXHTML export
Uwe and Georg have recently found and fixed a bug in our internal XHTML export. Why not test the export automatically? These tests would be fast (compared to the LaTeX compilation tests we have). I think our current XHTML tests just check that the exit code is zero. We can extend these tests to work as follows: 1. Export to .xhtml and check the exit code. 2. Check the terminal output from LyX (I think we do this for other tests, such as the lyx2lyx tests). 3. Use xmllint or something similar and check exit code. I know nothing about XHTML. Does anyone have experience with xmllint or any other tool? The only other tool I saw mentioned when searching is XMLStarlet. If someone does have experience, which tool do you recommend? Also, I'd be curious which options you think should be used. It seems to me that we would not want a very strict validation, and basically want to check whether the file is expected to open in a webbrowser without parser errors. Scott signature.asc Description: PGP signature
