Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Hello Ulrike,

> Which fatal clash?  Do you have an example?

Sorry, it is not a fatal crash, only a bunch of error messages and
invalid generated HTML file. I think I've posted this sample before,
but here it is to be sure:


\DocumentMetadata{testphase={phase-III,math}}
\documentclass{article}
\usepackage{graphicx}
\usepackage{amsmath}
\begin{document}
\section{Does section work?}

hello inline $a = b$.

\[
  a = \frac{b^2}{\sqrt{c}}
\]

\end{document}
%

You can compile it using:

$ make4ht -m draft -a debug sample.tex "mathml"

Without display math, it compiles without problems, so tagging doesn't
cause errors on itself, but only when you use commands that clash.

Best regards,
Michal

Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Ulrike Fischer]

Hallo Michal,

> I've disabled the \DocumentMetadata command in tex4ht.sty. Thich change needs
> the development version of make4ht because in the stable version, tex4ht.sty
> is not loaded before \documentclass, where \DocumentMetadata should be used.
> The options passed to this command are saved in the \:DocumentMetadata
> command, so it should be possible to process them later.

> This change is necessary to prevent a fatal clash between LaTeX's tagging
> support and TeX4ht. 

Which fatal clash?  Do you have an example?

Ulrike

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #13, bug #477 (project tex4ht):

I've disabled the \DocumentMetadata command in tex4ht.sty. Thich change needs
the development version of make4ht because in the stable version, tex4ht.sty
is not loaded before \documentclass, where \DocumentMetadata should be used.
The options passed to this command are saved in the \:DocumentMetadata
command, so it should be possible to process them later.

This change is necessary to prevent a fatal clash between LaTeX's tagging
support and TeX4ht. 

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Ulrike Fischer]

>> Well I know that rewriting such patches is hard, but this \document
>> redefinition looks very fragile. Sadly I can't make a concrete suggestion
>> how to change it as I have no idea what all the code is doing and what
>> should come before and after other things.

> Yes, it will be fragile, unfortunately. This is the code that loads
> .4ht files and then calls the original \document:

[...]

> It checks if the .cfg file was required, if it wasn't, it will call
> \Preamble, \begin{document}, and \EndPreamble. 

[...]

I know, I mean I checked the code and see the details. What I do not 
understand is the overall picture. 

So if you do a step back and remove in your mind all this 
redefinitions and assume a fresh, unchanged \document command. 

Which tasks do you have to do when you encounter \begin{document} and
which when you encounter \document? 
Which of them can you do with the normal hooks? 
Which of them can't you do and why not? What fails?

Ulrike

Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

>
> This won't fix the error with \DocumentMetadata as the
> testphase code loads there amsmath not with the standard hooks but in
> our own hook:
>
> \tl_gput_right:Nn \@kernel@before@begindocument
>   { \RequirePackage { amsmath } }
>
> (which doesn't look right, but moving it to
> \@kernel@after@begindocument@before doesn't work either).
>
> We could probably load amsmath earlier, but naturally the problem
> could reappear with other packages or code.
>

when I tried to call this command together with the other two hooks in
tex4ht.sty, it produced a fatal error. Amsmath was loaded, but there
were other errors related to the tagging code.

I will need to investigate it more, but the only thing that I found to
work so far is to require tex4ht.sty right at the beginning of the
processing, and to redefine \DocumentMetadata to do nothing.

>
> Well I know that rewriting such patches is hard, but this \document
> redefinition looks very fragile. Sadly I can't make a concrete suggestion
> how to change it as I have no idea what all the code is doing and what
> should come before and after other things.

Yes, it will be fragile, unfortunately. This is the code that loads
.4ht files and then calls the original \document:


\ifx \config:file\empty
   \def\:temp{\ifx \config:opt\:UnDef
   \edef\config:opt{\ifx \config:file\empty html\fi}\fi
\expandafter\Preamble\expandafter{\config:opt
   }\csname begin\endcsname{document}\EndPreamble
}%
\else
   \def\:temp{\input \config:file\relax
}%
\fi \:temp

It checks if the .cfg file was required, if it wasn't, it will call
\Preamble, \begin{document}, and \EndPreamble. The config file needs
to call these commands too, but it can also contain \Configure
commands used before \begin{document}, so it depends on the fact that
.4ht files should be loaded at that moment. I've also tried to call
original \begin{document} before this code block, but it produced more
errors. So again, something to investigate more.

Best regards,
Michal

Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Ulrike Fischer]

>> while there certainly can/will be problems, this error here seems to be
>> unrelated to \DocumentMetadata. I get an error with make4ht also for this:
>>

> thanks for this example. You are right. It seems that packages added
> in \AddToHook{begindocument/before} or \AtBeginDocument are ignored
> too. It is strange because I thought .4ht files captured them, but
> they are not.

> I was able to fix these hooks issues by manually calling:

> \ifdefined\UseHook
> \UseHook {begindocument/before}
> \UseHook {begindocument}
> \RemoveFromHook{begindocument/before}
> \RemoveFromHook{begindocument}
> \fi

This won't fix the error with \DocumentMetadata as the 
testphase code loads there amsmath not with the standard hooks but in 
our own hook:

\tl_gput_right:Nn \@kernel@before@begindocument
  { \RequirePackage { amsmath } }
  
(which doesn't look right, but moving it to 
\@kernel@after@begindocument@before doesn't work either). 

We could probably load amsmath earlier, but naturally the problem 
could reappear with other packages or code.

> in redefinition of the \document command in tex4ht.sty. I wonder if it
> wouldn't break something else, as in the original version of
> \document, the begindocument hook is called much later, after setting
> various variables. It can break some things in theory. However, the
> variables are mainly related to line width and similar stuff, which
> don't matter much in the conversion process.


Well I know that rewriting such patches is hard, but this \document
redefinition looks very fragile. Sadly I can't make a concrete suggestion
how to change it as I have no idea what all the code is doing and what
should come before and after other things.

Ulrike

Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Hello Ulrike,

> while there certainly can/will be problems, this error here seems to be
> unrelated to \DocumentMetadata. I get an error with make4ht also for this:
>

thanks for this example. You are right. It seems that packages added
in \AddToHook{begindocument/before} or \AtBeginDocument are ignored
too. It is strange because I thought .4ht files captured them, but
they are not.

I was able to fix these hooks issues by manually calling:

\ifdefined\UseHook
\UseHook {begindocument/before}
\UseHook {begindocument}
\RemoveFromHook{begindocument/before}
\RemoveFromHook{begindocument}
\fi

in redefinition of the \document command in tex4ht.sty. I wonder if it
wouldn't break something else, as in the original version of
\document, the begindocument hook is called much later, after setting
various variables. It can break some things in theory. However, the
variables are mainly related to line width and similar stuff, which
don't matter much in the conversion process.

It doesn't fix the issue with \DocumentMetadata, but that can be fixed
by loading tex4ht.sty earlier by make4ht.

Best regards,
Michal

Re: [tex4ht] [bug #477] Support for new LaTeX hook management

[Ulrike Fischer]

Hello Michal,

> We will need to make some changes to TeX4ht sources to support the metadata
> block used before \documentclass, because TeX4ht is inactive at that moment,
> so it doesn't track loaded files. It caused problems with my tests, namely
> because it loaded the Amsmath package. There can be potentially much more
> problems.

while there certainly can/will be problems, this error here seems to be 
unrelated to \DocumentMetadata. I get an error with make4ht also for this:

\documentclass{article}

\AddToHook{begindocument/before}{\RequirePackage{amsmath}}

\begin{document}
abc
\end{document}


Ulrike

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #12, bug #477 (project tex4ht):

There is some discussion about the new LaTeX tagging project and TeX4ht:
https://github.com/latex3/latex2e/discussions/1019 

We will need to make some changes to TeX4ht sources to support the metadata
block used before \documentclass, because TeX4ht is inactive at that moment,
so it doesn't track loaded files. It caused problems with my tests, namely
because it loaded the Amsmath package. There can be potentially much more
problems.

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Karl Berry]

Follow-up Comment #11, bug #477 (project tex4ht):

Most excellent, Michal. I pushed everything to TL, r63816.

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #10, bug #477 (project tex4ht):

Karl found that the a recent LaTeX kernel update broke TeX4ht support for  the
\chapter command. We got compilation errors about missing \endcsname when it
was used. 

I've found that the issue was caused by TeX4ht redefinition of the
\MakeUppercase command. It was a quite old code, and for some reason, it
handled also the \@tabacckludge command, which is used for accented
characters. It seems that the recent LaTeX update broke it completely, so even
the test uppercasing stopped to work. For this reason, I've removed these
redefinitions completely. I think that the reason why this code was used no
longer applies, as accented words work fine even after the removal. 



___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #9, bug #477 (project tex4ht):

I am posting this just as a reminder that new features are coming to LaTeX:
https://www.latex-project.org//news/2022/03/21/latex-dev-2022-1/

Some notable changes:

- new handling of utf-8 characters in PDFLaTeX
- new interface for declaring document metadata

Ad characters: I've tested some basic accents used in Czech, and it seems that
it works well. So I hope that nothing breaks in this regard.

Ad metadata: this could be useful for getting metadata that are required by
some XML formats, as well as for HTML






___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #8, bug #477 (project tex4ht):

I've found that we are missing support for \AddToHook{env/name/before} and
other hooks for environments. I've just fixed that in the sources.

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #7, bug #477 (project tex4ht):

The big update of LaTeX kernel is now in TeX Live. It seems that it didn't
broke TeX4ht, which is nice :)

Here are the news:
https://www.latex-project.org/news/latex2e-news/ltnews32.pdf


___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #6, bug #477 (project tex4ht):

It seems that patching depended on exact definition of \@textsuperscript and
\@textsubscript in LaTeX core. I don't know why, probably for some historical
reasons. I've removed the conditions and it seems to work in LaTeX-dev and
normal LaTeX without issues.

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #5, bug #477 (project tex4ht):

It seems that also \textsuperscript and \textsubscript is missing in
LaTeX-dev. Respectivelly that we need to patch different macros.

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Karl Berry]

Follow-up Comment #4, bug #477 (project tex4ht):

It might be worth asking the latexers for advice. Maybe they have some idea of
how to profile. Since tex4ht didn't change (I presume), the slowdown in latex
is surely not desirable from their point of view either.

Also, maybe try running just one of the latex invocations, instead of the
whole make4ht or htlatex sequence. That seems like what would be needed to
debug, anyway.


___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Follow-up Comment #3, bug #477 (project tex4ht):

I've installed the new LaTeX-dev update and found that the use of Fontspec in
a document caused fatal error. It was just necessary to remove
\RequirePackage{expl3} from usepackage-fontspec.4ht and it works again.

One thing I noticed is that the compilation is much slower with LaTeX-dev. My
simple test document went from 4 seconds to almost 8 :/ 

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Update of bug #477 (project tex4ht):

 Open/Closed:  Closed => Open   

___

Follow-up Comment #2:

We will also need to support new hooks in environment handling and probably
others. It should be handled before hooks are added to the main LaTeX kernel.

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

Update of bug #477 (project tex4ht):

 Open/Closed:Open => Closed 

___

Follow-up Comment #1:

I've updated TeX4ht sources for the upcoming changes. I want to thank Frank
Mittelbach for help and suggestions, it would take me much longer to find the
solution.

In the future, we will need to adapt more TeX4ht functionality for the new
hook system, especially the LaTeX environment handling. 

___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/

[tex4ht] [bug #477] Support for new LaTeX hook management

[Michal Hoftich]

URL:
  

 Summary: Support for new LaTeX hook management
 Project: tex4ht
Submitted by: michal_h21
Submitted on: Thu 23 Jul 2020 04:24:20 PM EEST
Category: None
Priority: 5 - Normal
Severity: 5 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any

___

Details:

The development version of LaTeX changes some internal code related to the
hook management. It will be more adjustable, but it can break older code. As
you may suspect, it breaks TeX4ht. 

See this thread for details:
https://listserv.uni-heidelberg.de/cgi-bin/wa?A2=LATEX-L;88a2d211.2007=

I think we identified the issues, thanks to Joseph Wright and Frank
Mittelbach. I've already found fixes, but we will make more tests and try to
identify the potential issues with the new implementation.




___

Reply to this item at:

  

___
  Message sent via/by Puszcza
  http://puszcza.gnu.org.ua/