[PATCH] Make the export of macros thread-safer

[Vincent van Ravesteijn]

In general, the metrics() functions should not change the Buffer; we have 
updateBuffer/updateMacros for that purpose.
---
 src/mathed/MathMacroTemplate.cpp |8 ++--
 src/mathed/MathMacroTemplate.h   |4 ++--
 2 files changed, 4 insertions(+), 8 deletions(-)

diff --git a/src/mathed/MathMacroTemplate.cpp b/src/mathed/MathMacroTemplate.cpp
index e82de01..62c681b 100644
--- a/src/mathed/MathMacroTemplate.cpp
+++ b/src/mathed/MathMacroTemplate.cpp
@@ -465,7 +465,7 @@ docstring MathMacroTemplate::name() const
 }
 
 
-void MathMacroTemplate::updateToContext(MacroContext const & mc) const
+void MathMacroTemplate::updateToContext(MacroContext const & mc)
 {
redefinition_ = mc.get(name()) != 0;
 }
@@ -551,13 +551,9 @@ void MathMacroTemplate::metrics(MetricsInfo & mi, 
Dimension & dim) const
 
// valid macro?
MacroData const * macro = 0;
-   if (validName()) {
+   if (validName())
macro = mi.macrocontext.get(name());
 
-   // updateToContext() - avoids another lookup
-   redefinition_ = macro != 0;
-   }
-
// update look?
int argsInDef = maxArgumentInDefinition();
if (lookOutdated_ || argsInDef != argsInLook_) {
diff --git a/src/mathed/MathMacroTemplate.h b/src/mathed/MathMacroTemplate.h
index 61f82e9..0bb3d4d 100644
--- a/src/mathed/MathMacroTemplate.h
+++ b/src/mathed/MathMacroTemplate.h
@@ -87,7 +87,7 @@ public:
virtual void validate(LaTeXFeatures &) const;
 
/// decide whether its a redefinition
-   void updateToContext(MacroContext const & mc) const;
+   void updateToContext(MacroContext const & mc);
 
///
void draw(PainterInfo & pi, int x, int y) const;
@@ -164,7 +164,7 @@ private:
/// (re)newcommand or def
mutable MacroType type_;
/// defined before already?
-   mutable bool redefinition_;
+   bool redefinition_;
///
void createLook(int args) const;
///
-- 
1.7.9.5

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

On Wed, Oct 9, 2013 at 10:22 PM, Georg Baum
georg.b...@post.rwth-aachen.dewrote:

 Enrico Forestieri wrote:

  On Tue, Oct 08, 2013 at 09:49:44PM +0200, Vincent van Ravesteijn wrote:
 
  I tried to use QThreadStorage, which works on Windows, but on Linux
  I keep getting segmentation faults.

 Vincent, I extended your patch to protect all IconvProcessor instances.
 Thanks to Enricos documentation links I am now also sure that the mutex in
 convert() is not needed, so I removed it. Using thread-local storage has
 also the advantage that we can share one IconvProcessor instance per thread
 for each encoding pair. I believe that this patch makes all iconv
 conversions 100% thread-safe. I did not see any segfault on Debian Wheezy,
 and I could not get any wrong conversion. What do others think?

 This patch works for me as well now.

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

The other difference I noticed is that on Cygwin and Solaris I get
 \renewcommand instead of \newcommand. That is, applying your patch,
 on Debian exporting from the GUI or from command line, produces the
 same result, but lyx segafaults on exit.
 On Cygwin and Solaris, exporting from the GUI changes \newcommand
 into \renewcommand, but lyx quits normally.
 The joy of multithreading, I suppose.


I guess the problem is that the class MacroData has a lot of mutable
variables that are being changed from const metrics() and write() functions.

Vincent

Re: Datalosses

[Vincent van Ravesteijn]

Op 4-10-2013 10:51, Jürgen Spitzmüller schreef:

Pavel Sanda wrote:

It was trimmed or with garbage like in #8854?

Trimmed. As said, it stopped at the beginning of a tabular inset. No garbage.

Jürgen

Hi Jurgen,

How do you feel about this ?

Time to investigate further, or is it relatively safe to move forward.

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

On Wed, Oct 9, 2013 at 10:22 PM, Georg Baum
<georg.b...@post.rwth-aachen.de>wrote:

> Enrico Forestieri wrote:
>
> > On Tue, Oct 08, 2013 at 09:49:44PM +0200, Vincent van Ravesteijn wrote:
> >>
> >> I tried to use QThreadStorage, which works on Windows, but on Linux
> >> I keep getting segmentation faults.
>
> Vincent, I extended your patch to protect all IconvProcessor instances.
> Thanks to Enricos documentation links I am now also sure that the mutex in
> convert() is not needed, so I removed it. Using thread-local storage has
> also the advantage that we can share one IconvProcessor instance per thread
> for each encoding pair. I believe that this patch makes all iconv
> conversions 100% thread-safe. I did not see any segfault on Debian Wheezy,
> and I could not get any wrong conversion. What do others think?
>
> This patch works for me as well now.

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

The other difference I noticed is that on Cygwin and Solaris I get
> \renewcommand instead of \newcommand. That is, applying your patch,
> on Debian exporting from the GUI or from command line, produces the
> same result, but lyx segafaults on exit.
> On Cygwin and Solaris, exporting from the GUI changes \newcommand
> into \renewcommand, but lyx quits normally.
> The joy of multithreading, I suppose.
>

I guess the problem is that the class MacroData has a lot of mutable
variables that are being changed from const metrics() and write() functions.

Vincent

Re: Datalosses

[Vincent van Ravesteijn]

Op 4-10-2013 10:51, Jürgen Spitzmüller schreef:

Pavel Sanda wrote:

It was trimmed or with garbage like in #8854?

Trimmed. As said, it stopped at the beginning of a tabular inset. No garbage.

Jürgen

Hi Jurgen,

How do you feel about this ?

Time to investigate further, or is it relatively safe to move forward.

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 8-10-2013 14:30, Jean-Marc Lasgouttes schreef:

08/10/2013 14:24, Jean-Marc Lasgouttes:

07/10/2013 23:24, Georg Baum:

I don't remember the mutex story, however the problem is surely due to
the static output buffer in iconv_convert. The question is: why was it
made static?


For performance reasons (see r14854), and I believe that removing the
static
keyword would be quite visible in the profiler.


Is there something like per-thread static storage? We do not really need
mutexes, since it is not concurrent access to the same information.


For example, is the following useful?
http://www.boost.org/doc/libs/1_35_0/doc/html/thread/thread_local_storage.html 



JMarc


I tried to use QThreadStorage, which works on Windows, but on Linux I 
keep getting segmentation faults.


http://git.lyx.org/?p=developers/vfr/lyx.git;a=blobdiff;f=src/support/unicode.cpp;h=e8ede4f086c47101c5e34ce0ff6e7b3da6bb477c;hp=95415a5538e45ad11a4ed4695c5b277ae79024c3;hb=9e0ef6d9e11d6d31ec3a6e271266563e4089703b;hpb=c6c07d28ae99a0dee5fc4b76aae2827d3d09cf93 



Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 8-10-2013 14:30, Jean-Marc Lasgouttes schreef:

08/10/2013 14:24, Jean-Marc Lasgouttes:

07/10/2013 23:24, Georg Baum:

I don't remember the mutex story, however the problem is surely due to
the static output buffer in iconv_convert. The question is: why was it
made static?


For performance reasons (see r14854), and I believe that removing the
static
keyword would be quite visible in the profiler.


Is there something like per-thread static storage? We do not really need
mutexes, since it is not concurrent access to the same information.


For example, is the following useful?
http://www.boost.org/doc/libs/1_35_0/doc/html/thread/thread_local_storage.html 



JMarc


I tried to use QThreadStorage, which works on Windows, but on Linux I 
keep getting segmentation faults.


http://git.lyx.org/?p=developers/vfr/lyx.git;a=blobdiff;f=src/support/unicode.cpp;h=e8ede4f086c47101c5e34ce0ff6e7b3da6bb477c;hp=95415a5538e45ad11a4ed4695c5b277ae79024c3;hb=9e0ef6d9e11d6d31ec3a6e271266563e4089703b;hpb=c6c07d28ae99a0dee5fc4b76aae2827d3d09cf93 



Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 6-10-2013 18:09, Richard Heck schreef:


I just wanted to call everyone's attention to this bug:
http://www.lyx.org/trac/ticket/8854
It appears to be a very nasty dataloss bug in 2.1-beta1. What seems to 
be happening is that, on save, the LyX file is corrupted in very weird 
ways, with random parts of the file being replaced by things like 
mathnorma or Plain La---fragments of things that do make sense in 
other contexts.


A report of what looks like the same bug in 2.0.6 is here:
http://tex.stackexchange.com/questions/134009/lyx-inserts-and-removes-words-randomly 



This was another strange bug we which sounds similar in a way and we 
apparently never solved it:

http://comments.gmane.org/gmane.editors.lyx.general/78971

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 7-10-2013 21:05, Enrico Forestieri schreef:

On Mon, Oct 07, 2013 at 02:28:18PM -0400, Richard Heck wrote:

It is very strange that I can't reproduce, if this is the cause,
and, indeed, that people haven't been seeing this bug all along.

Actually, all my files are converted correctly. But using the one
attached to the bug report triggers reproducibly the behavior.
Not that I get converted files which are always mangled in different
ways, which is a symptom of a race condition. Also, I have some large
files (~ 700k) that convert flawlessly. So, the bug also depends on
the structure of the document, somehow.

When I open the document in a plain text editor, it complains about 
characters that cannot be represented in the current codeset (or 
something like that).


I'm now looking at commit d5a41946a (Jurgen Spitzmuller; May 19 2009).

In this code the static char array is replaced by a static vector. This 
vector is resized when maxoutbufsize is larger than the current size of 
the vector. However, once the size of this vector is increased, it will 
remain this large for ever.


Maybe the old code always overwrites the full variable, while this new 
code does not.


Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 7-10-2013 21:05, Enrico Forestieri schreef:
So, the bug also depends on the structure of the document, somehow. 


Is the registered trademark symbol on line 460 and 480 of the lyx file 
the problem maybe ?


Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 7-10-2013 21:18, Vincent van Ravesteijn schreef:

Op 7-10-2013 21:05, Enrico Forestieri schreef:

On Mon, Oct 07, 2013 at 02:28:18PM -0400, Richard Heck wrote:

It is very strange that I can't reproduce, if this is the cause,
and, indeed, that people haven't been seeing this bug all along.

Actually, all my files are converted correctly. But using the one
attached to the bug report triggers reproducibly the behavior.
Not that I get converted files which are always mangled in different
ways, which is a symptom of a race condition. Also, I have some large
files (~ 700k) that convert flawlessly. So, the bug also depends on
the structure of the document, somehow.

When I open the document in a plain text editor, it complains about 
characters that cannot be represented in the current codeset (or 
something like that).


I'm now looking at commit d5a41946a (Jurgen Spitzmuller; May 19 2009).

In this code the static char array is replaced by a static vector. 
This vector is resized when maxoutbufsize is larger than the current 
size of the vector. However, once the size of this vector is 
increased, it will remain this large for ever.


Maybe the old code always overwrites the full variable, while this new 
code does not.



I could reproduce it now a few times. To do this:
- I switched to the Czech UI of Ubuntu,
- I replaced the default buffer size to 0.
I don't know whether the above steps are required or not.

Something else I notice: Sometime the macros are exported as 
\newcommand and sometimes as \renewcommand. This seems to be rather 
random.


Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 6-10-2013 18:09, Richard Heck schreef:


I just wanted to call everyone's attention to this bug:
http://www.lyx.org/trac/ticket/8854
It appears to be a very nasty dataloss bug in 2.1-beta1. What seems to 
be happening is that, on save, the LyX file is corrupted in very weird 
ways, with random parts of the file being replaced by things like 
"mathnorma" or "Plain La"---fragments of things that do make sense in 
other contexts.


A report of what looks like the same bug in 2.0.6 is here:
http://tex.stackexchange.com/questions/134009/lyx-inserts-and-removes-words-randomly 



This was another strange bug we which sounds similar in a way and we 
apparently never solved it:

http://comments.gmane.org/gmane.editors.lyx.general/78971

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 7-10-2013 21:05, Enrico Forestieri schreef:

On Mon, Oct 07, 2013 at 02:28:18PM -0400, Richard Heck wrote:

It is very strange that I can't reproduce, if this is the cause,
and, indeed, that people haven't been seeing this bug all along.

Actually, all my files are converted correctly. But using the one
attached to the bug report triggers reproducibly the behavior.
Not that I get converted files which are always mangled in different
ways, which is a symptom of a race condition. Also, I have some large
files (~ 700k) that convert flawlessly. So, the bug also depends on
the structure of the document, somehow.

When I open the document in a plain text editor, it complains about 
characters that cannot be represented in the current codeset (or 
something like that).


I'm now looking at commit d5a41946a (Jurgen Spitzmuller; May 19 2009).

In this code the static char array is replaced by a static vector. This 
vector is resized when maxoutbufsize is larger than the current size of 
the vector. However, once the size of this vector is increased, it will 
remain this large for ever.


Maybe the old code always overwrites the full variable, while this new 
code does not.


Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 7-10-2013 21:05, Enrico Forestieri schreef:
So, the bug also depends on the structure of the document, somehow. 


Is the registered trademark symbol on line 460 and 480 of the lyx file 
the problem maybe ?


Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 7-10-2013 21:18, Vincent van Ravesteijn schreef:

Op 7-10-2013 21:05, Enrico Forestieri schreef:

On Mon, Oct 07, 2013 at 02:28:18PM -0400, Richard Heck wrote:

It is very strange that I can't reproduce, if this is the cause,
and, indeed, that people haven't been seeing this bug all along.

Actually, all my files are converted correctly. But using the one
attached to the bug report triggers reproducibly the behavior.
Not that I get converted files which are always mangled in different
ways, which is a symptom of a race condition. Also, I have some large
files (~ 700k) that convert flawlessly. So, the bug also depends on
the structure of the document, somehow.

When I open the document in a plain text editor, it complains about 
characters that cannot be represented in the current codeset (or 
something like that).


I'm now looking at commit d5a41946a (Jurgen Spitzmuller; May 19 2009).

In this code the static char array is replaced by a static vector. 
This vector is resized when maxoutbufsize is larger than the current 
size of the vector. However, once the size of this vector is 
increased, it will remain this large for ever.


Maybe the old code always overwrites the full variable, while this new 
code does not.



I could reproduce it now a few times. To do this:
- I switched to the Czech UI of Ubuntu,
- I replaced the default buffer size to 0.
I don't know whether the above steps are required or not.

Something else I notice: Sometime the macros are exported as 
"\newcommand" and sometimes as "\renewcommand". This seems to be rather 
random.


Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 6-10-2013 18:09, Richard Heck schreef:


I just wanted to call everyone's attention to this bug:
http://www.lyx.org/trac/ticket/8854
It appears to be a very nasty dataloss bug in 2.1-beta1. What seems to 
be happening is that, on save, the LyX file is corrupted in very weird 
ways, with random parts of the file being replaced by things like 
mathnorma or Plain La---fragments of things that do make sense in 
other contexts.


A report of what looks like the same bug in 2.0.6 is here:
http://tex.stackexchange.com/questions/134009/lyx-inserts-and-removes-words-randomly 

That was on Windows, as well, whereas #8854 is on Ubuntu. Another 
difference is that #8854 seems to show up only on export. The original 
report was of corruption in the exported 2.0 file, but by looking in 
the temporary directory we found that the 2.1 file was already 
corrupt. That makes it sound as if the problem has to do with 
background export. But the 2.0.6 report seems to concern just saving 
the file and not export.


The reporter has done some excellent work to help us figure this out, 
but neither Pavel nor I have been able to reproduce---though the 
reporter sees it all the time. It would really, really help if someone 
could manage to reproduce this.


Obviously, it would also help if someone has a good idea what might 
cause this.


Richard



Is it really a blocking bug if it is only seen by one person who only 
can see this bug when running in a multi-processor virtual machine ?


This can really take ages before we figure out what might be happening.

Vincent

Re: 2.1.0 Blocker

[Vincent van Ravesteijn]

Op 6-10-2013 18:09, Richard Heck schreef:


I just wanted to call everyone's attention to this bug:
http://www.lyx.org/trac/ticket/8854
It appears to be a very nasty dataloss bug in 2.1-beta1. What seems to 
be happening is that, on save, the LyX file is corrupted in very weird 
ways, with random parts of the file being replaced by things like 
"mathnorma" or "Plain La"---fragments of things that do make sense in 
other contexts.


A report of what looks like the same bug in 2.0.6 is here:
http://tex.stackexchange.com/questions/134009/lyx-inserts-and-removes-words-randomly 

That was on Windows, as well, whereas #8854 is on Ubuntu. Another 
difference is that #8854 seems to show up only on export. The original 
report was of corruption in the exported 2.0 file, but by looking in 
the temporary directory we found that the 2.1 file was already 
corrupt. That makes it sound as if the problem has to do with 
background export. But the 2.0.6 report seems to concern just saving 
the file and not export.


The reporter has done some excellent work to help us figure this out, 
but neither Pavel nor I have been able to reproduce---though the 
reporter sees it all the time. It would really, really help if someone 
could manage to reproduce this.


Obviously, it would also help if someone has a good idea what might 
cause this.


Richard



Is it really a blocking bug if it is only seen by one person who only 
can see this bug when running in a multi-processor virtual machine ?


This can really take ages before we figure out what might be happening.

Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 30-9-2013 17:49, Jean-Marc Lasgouttes schreef:

29/09/2013 17:57, Vincent van Ravesteijn:

Op 17-8-2013 11:38, Jürgen Spitzmüller schreef:

It stops at .deps/AppleSpellChecker.Po and other .deps/*.Po which are
unknown.

Jürgen

This still fails for me.

Is anyone around who knows how to fix this ?


I did some searching and, amazingly, there is no real solution that 
works with old and recent automake versions. It seems that the support 
for automake 1.11 to 1.14. Look for the answer of the maintainer to 
this same problem:

http://lists.gnu.org/archive/html/automake/2013-09/msg00012.html

From what I read, everything is supposed to work with automake 1.14. 
Can someone check that? I am amazed that such a basic functionality is 
still broken so long after subdir-objects has been introduced.


If it works, we can adapt the trick used in libcurl:

m4_ifdef([_AM_PROG_CC_C_O],
[
AM_INIT_AUTOMAKE([subdir-objects])
],[
AM_INIT_AUTOMAKE
])dnl

This allows to use subdir-objects only with version 1.14, where this 
_AM_PROG_CC_C_O got introduced.


Other wise, the patch posted later by Vincent makes sense, but it 
should really be marked by a big comment, because it make break 
something else later. In libcURL, they have a very nasty-looking 
distclean rewriter that modifies the code in place. I do not think we 
want to do that :)

https://github.com/bagder/curl/blob/master/m4/xc-am-iface.m4

JMarc


So, I will need to revert the subdir-objects option.

Apparently, there will be no problem until automake 2.0 ?

Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 30-9-2013 17:49, Jean-Marc Lasgouttes schreef:

29/09/2013 17:57, Vincent van Ravesteijn:

Op 17-8-2013 11:38, Jürgen Spitzmüller schreef:

It stops at .deps/AppleSpellChecker.Po and other .deps/*.Po which are
unknown.

Jürgen

This still fails for me.

Is anyone around who knows how to fix this ?


I did some searching and, amazingly, there is no real solution that 
works with old and recent automake versions. It seems that the support 
for automake 1.11 to 1.14. Look for the answer of the maintainer to 
this same problem:

http://lists.gnu.org/archive/html/automake/2013-09/msg00012.html

From what I read, everything is supposed to work with automake 1.14. 
Can someone check that? I am amazed that such a basic functionality is 
still broken so long after subdir-objects has been introduced.


If it works, we can adapt the trick used in libcurl:

m4_ifdef([_AM_PROG_CC_C_O],
[
AM_INIT_AUTOMAKE([subdir-objects])
],[
AM_INIT_AUTOMAKE
])dnl

This allows to use subdir-objects only with version 1.14, where this 
_AM_PROG_CC_C_O got introduced.


Other wise, the patch posted later by Vincent makes sense, but it 
should really be marked by a big comment, because it make break 
something else later. In libcURL, they have a very nasty-looking 
distclean rewriter that modifies the code in place. I do not think we 
want to do that :)

https://github.com/bagder/curl/blob/master/m4/xc-am-iface.m4

JMarc


So, I will need to revert the subdir-objects option.

Apparently, there will be no problem until automake 2.0 ?

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 27-9-2013 23:53, Stephan Witt schreef:

Regarding the details:
* BufferEncoding.cpp is a manual copy of Encoding.cpp
- that's why the wrong comment
- that's why the name of the file
- that's why the superfluous includes
* CharInfo was a private struct
- I've made a class to make it more functional but didn't want to make it public
- It should be used inside Encoding(s) only
- Yes, one can move it to a separate public class with header file
- Yes, that's why the static methods in Encoding.cpp …

Who is we?

Well, I have no other choice than to fix this myself in the coming days and to 
prepare the release of beta 2 in the weekend, or to reject this branch for now 
and prepare the release of beta 2.

I did it myself.


Thank you, I've merged the branch now, after that I splitted and merged 
some commits in more logical building blocks. At least what seemed 
logical in my opinion.


You will have to cherry-pick the commit use 
com.apple.compilers.llvm.clang.1_0 for newer Xcode versions yourself, 
because it didn't seem to be related to killing the TEX2LYX define.


Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 17-8-2013 11:38, Jürgen Spitzmüller schreef:

It stops at .deps/AppleSpellChecker.Po and other .deps/*.Po which are unknown.

Jürgen

This still fails for me.

Is anyone around who knows how to fix this ?

Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 29-9-2013 18:50, Jürgen Spitzmüller schreef:

Vincent van Ravesteijn wrote:

This still fails for me.

Is anyone around who knows how to fix this ?

Last time, removing subdir-objects from AM_INIT_AUTOMAKE in configure.ac
helped.

Jürgen


Well, I just added this again.

So.. does this mean we have to remove it .. again ?

Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 17-8-2013 11:38, Jürgen Spitzmüller schreef:

It stops at .deps/AppleSpellChecker.Po and other .deps/*.Po which are unknown.

Jürgen
The culprit seems to be the following line in the output of make 
distclean:


rm -rf ../.deps ../graphics/.deps ../insets/.deps .deps

while being in the directory src/tex2lyx.

This means that when we continue, we suddenly miss the .deps directories.

Vincent

[PATCH] Fix make distclean

[Vincent van Ravesteijn]

This only cleans the .deps directories that won't be needed later on.
---

This fixes it for me. Any comments on this ?

Vincent

 src/support/Makefile.am |4 
 src/tex2lyx/Makefile.am |4 
 2 files changed, 8 insertions(+)

diff --git a/src/support/Makefile.am b/src/support/Makefile.am
index 708d19d..e82761d 100644
--- a/src/support/Makefile.am
+++ b/src/support/Makefile.am
@@ -30,6 +30,10 @@ liblyxsupport_a_DEPENDENCIES = $(MOCEDFILES)
 AM_CPPFLAGS += $(PCH_FLAGS) -I$(srcdir)/.. $(BOOST_INCLUDES)
 AM_CPPFLAGS += $(QT4_CPPFLAGS) $(QT4_INCLUDES)
 
+distclean: distclean-am
+   -rm -rf ./$(DEPDIR) linkback/$(DEPDIR) mythes/$(DEPDIR)
+   -rm -f Makefile
+
 liblyxsupport_a_SOURCES = \
FileMonitor.h \
FileMonitor.cpp \
diff --git a/src/tex2lyx/Makefile.am b/src/tex2lyx/Makefile.am
index 0aa887e..4388054 100644
--- a/src/tex2lyx/Makefile.am
+++ b/src/tex2lyx/Makefile.am
@@ -66,6 +66,10 @@ alltests-recursive: tex2lyx
 updatetests: tex2lyx
$(PYTHON) $(srcdir)/test/runtests.py ./tex2lyx 
$(top_srcdir)/lib/scripts $(srcdir)/test
 
+distclean: distclean-am
+   -rm -rf ./$(DEPDIR)
+   -rm -f Makefile
+
 LINKED_FILES = \
../Author.cpp \
../Color.cpp \
-- 
1.7.9.5

Re: [PATCH] Fix make distclean

[Vincent van Ravesteijn]

Op 29-9-2013 22:16, Stephan Witt schreef:

Am 29.09.2013 um 21:48 schrieb Pavel Sanda sa...@lyx.org:


Stephan Witt wrote:

I cannot imagine how a rm -f can fail, so the leading minus is not needed.

If DEPDIR doesn't exists it will fail. P

No. It will not fail. That's the effect of -f.

Stephan


I copied it from Makefile.in, so I guess it will be ok to leave it 
there, whether it can or cannot fail in practice.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 27-9-2013 23:53, Stephan Witt schreef:

Regarding the details:
* BufferEncoding.cpp is a manual copy of Encoding.cpp
- that's why the wrong comment
- that's why the name of the file
- that's why the superfluous includes
* CharInfo was a private struct
- I've made a class to make it more functional but didn't want to make it public
- It should be used inside Encoding(s) only
- Yes, one can move it to a separate public class with header file
- Yes, that's why the static methods in Encoding.cpp …

Who is we?

Well, I have no other choice than to fix this myself in the coming days and to 
prepare the release of beta 2 in the weekend, or to reject this branch for now 
and prepare the release of beta 2.

I did it myself.


Thank you, I've merged the branch now, after that I splitted and merged 
some commits in more logical building blocks. At least what seemed 
logical in my opinion.


You will have to cherry-pick the commit "use 
com.apple.compilers.llvm.clang.1_0 for newer Xcode versions" yourself, 
because it didn't seem to be related to killing the TEX2LYX define.


Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 17-8-2013 11:38, Jürgen Spitzmüller schreef:

It stops at .deps/AppleSpellChecker.Po and other .deps/*.Po which are unknown.

Jürgen

This still fails for me.

Is anyone around who knows how to fix this ?

Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 29-9-2013 18:50, Jürgen Spitzmüller schreef:

Vincent van Ravesteijn wrote:

This still fails for me.

Is anyone around who knows how to fix this ?

Last time, removing subdir-objects from AM_INIT_AUTOMAKE in configure.ac
helped.

Jürgen


Well, I just added this again.

So.. does this mean we have to remove it .. again ?

Vincent

Re: Make distclean failure

[Vincent van Ravesteijn]

Op 17-8-2013 11:38, Jürgen Spitzmüller schreef:

It stops at .deps/AppleSpellChecker.Po and other .deps/*.Po which are unknown.

Jürgen
The culprit seems to be the following line in the output of "make 
distclean":


rm -rf ../.deps ../graphics/.deps ../insets/.deps .deps

while being in the directory src/tex2lyx.

This means that when we continue, we suddenly miss the .deps directories.

Vincent

[PATCH] Fix make distclean

[Vincent van Ravesteijn]

This only cleans the .deps directories that won't be needed later on.
---

This fixes it for me. Any comments on this ?

Vincent

 src/support/Makefile.am |4 
 src/tex2lyx/Makefile.am |4 
 2 files changed, 8 insertions(+)

diff --git a/src/support/Makefile.am b/src/support/Makefile.am
index 708d19d..e82761d 100644
--- a/src/support/Makefile.am
+++ b/src/support/Makefile.am
@@ -30,6 +30,10 @@ liblyxsupport_a_DEPENDENCIES = $(MOCEDFILES)
 AM_CPPFLAGS += $(PCH_FLAGS) -I$(srcdir)/.. $(BOOST_INCLUDES)
 AM_CPPFLAGS += $(QT4_CPPFLAGS) $(QT4_INCLUDES)
 
+distclean: distclean-am
+   -rm -rf ./$(DEPDIR) linkback/$(DEPDIR) mythes/$(DEPDIR)
+   -rm -f Makefile
+
 liblyxsupport_a_SOURCES = \
FileMonitor.h \
FileMonitor.cpp \
diff --git a/src/tex2lyx/Makefile.am b/src/tex2lyx/Makefile.am
index 0aa887e..4388054 100644
--- a/src/tex2lyx/Makefile.am
+++ b/src/tex2lyx/Makefile.am
@@ -66,6 +66,10 @@ alltests-recursive: tex2lyx
 updatetests: tex2lyx
$(PYTHON) "$(srcdir)/test/runtests.py" ./tex2lyx 
"$(top_srcdir)/lib/scripts" "$(srcdir)/test"
 
+distclean: distclean-am
+   -rm -rf ./$(DEPDIR)
+   -rm -f Makefile
+
 LINKED_FILES = \
../Author.cpp \
../Color.cpp \
-- 
1.7.9.5

Re: [PATCH] Fix make distclean

[Vincent van Ravesteijn]

Op 29-9-2013 22:16, Stephan Witt schreef:

Am 29.09.2013 um 21:48 schrieb Pavel Sanda :


Stephan Witt wrote:

I cannot imagine how a "rm -f" can fail, so the leading minus is not needed.

If DEPDIR doesn't exists it will fail. P

No. It will not fail. That's the effect of "-f".

Stephan


I copied it from Makefile.in, so I guess it will be ok to leave it 
there, whether it can or cannot fail in practice.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

- The commit about splitting the encoding stuff and moving helper functions 
into a new class, should really have some explanation why this is needed, and 
why we would need a new class for this. The construction around CharInfo 
becomes rather strange. It is defined in Encoding.cpp, which indicates that it 
should not be used outside Encoding.cpp. It is now used outside this file in 
BufferEncoding.cpp. Because of this, there are a lot of static functions added 
to the Encoding class just to allow BufferEncoding to use CharInfo as an 
incomplete type. Moreover, most of these functions aren't even used.  Why is 
this moved into a new class ? What was the problem that needed to be solved ? 
Can't we solve it with a private header file ?

- The file src/BufferEncoding.cpp starts with a comment that it is Encoding.cpp

- The class name is BufferEncodings so why don't we call the file 
BufferEncodings.cpp ? (I know Encodings in Encoding.cpp.., but is that a good 
reason?)

- The following includes in BufferEncoding.cpp are not needed:

#include BufferList.h
#include Lexer.h
#include LyXRC.h
#include support/debug.h
#include support/gettext.h
#include support/FileName.h
#include support/textutils.h
#include support/unicode.h
#include boost/cstdint.hpp
#include sstream

The root problem is the fact Encoding(s) classes are used both by LyX and 
tex2lyx
and these classes are using the Buffer stuff.

This forces to link tex2lyx with the Buffer etc. making it another LyX binary
or split the classes somehow. Originally it was done by a dirty hack -
using the same code even without making a copy and compiling it twice with 
different
compiler flags. That's not nice. Neither with C nor with C++.

Nevertheless I'm standing on the shoulder of giants and want to change the 
code base
only slightly if possible. I cannot use modern tools like Eclipse for 
refactoring the
code.


Well the giants have made a mess in certain areas of the code. And I 
believe this is increasing the mess a bit.



Regarding the details:
* BufferEncoding.cpp is a manual copy of Encoding.cpp
- that's why the wrong comment
- that's why the name of the file
- that's why the superfluous includes
* CharInfo was a private struct
- I've made a class to make it more functional but didn't want to make it public
- It should be used inside Encoding(s) only
- Yes, one can move it to a separate public class with header file
- Yes, that's why the static methods in Encoding.cpp …

Who is we?


Well, I have no other choice than to fix this myself in the coming days 
and to prepare the release of beta 2 in the weekend, or to reject this 
branch for now and prepare the release of beta 2.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

- The commit about splitting the encoding stuff and moving helper functions 
into a new class, should really have some explanation why this is needed, and 
why we would need a new class for this. The construction around CharInfo 
becomes rather strange. It is defined in Encoding.cpp, which indicates that it 
should not be used outside Encoding.cpp. It is now used outside this file in 
BufferEncoding.cpp. Because of this, there are a lot of static functions added 
to the Encoding class just to allow BufferEncoding to use CharInfo as an 
incomplete type. Moreover, most of these functions aren't even used.  Why is 
this moved into a new class ? What was the problem that needed to be solved ? 
Can't we solve it with a private header file ?

- The file src/BufferEncoding.cpp starts with a comment that it is Encoding.cpp

- The class name is BufferEncodings so why don't we call the file 
BufferEncodings.cpp ? (I know Encodings in Encoding.cpp.., but is that a good 
reason?)

- The following includes in BufferEncoding.cpp are not needed:

#include "BufferList.h"
#include "Lexer.h"
#include "LyXRC.h"
#include "support/debug.h"
#include "support/gettext.h"
#include "support/FileName.h"
#include "support/textutils.h"
#include "support/unicode.h"
#include 
#include 

The root problem is the fact Encoding(s) classes are used both by LyX and 
tex2lyx
and these classes are using the Buffer stuff.

This forces to link tex2lyx with the Buffer etc. making it another LyX binary
or split the classes somehow. Originally it was done by a dirty hack -
using the same code even without making a copy and compiling it twice with 
different
compiler flags. That's not nice. Neither with C nor with C++.

Nevertheless I'm standing on the "shoulder of giants" and want to change the 
code base
only slightly if possible. I cannot use modern tools like Eclipse for 
refactoring the
code.


Well the "giants" have made a mess in certain areas of the code. And I 
believe this is increasing the mess a bit.



Regarding the details:
* BufferEncoding.cpp is a manual copy of Encoding.cpp
- that's why the wrong comment
- that's why the name of the file
- that's why the superfluous includes
* CharInfo was a private struct
- I've made a class to make it more functional but didn't want to make it public
- It should be used inside Encoding(s) only
- Yes, one can move it to a separate public class with header file
- Yes, that's why the static methods in Encoding.cpp …

Who is we?


Well, I have no other choice than to fix this myself in the coming days 
and to prepare the release of beta 2 in the weekend, or to reject this 
branch for now and prepare the release of beta 2.


Vincent

Re: LyX unit testing

[Vincent van Ravesteijn]

Op 19-9-2013 8:37, Hashini Senaratne schreef:

Hello Vincent,


But in general, I would like to introduce unit testing, yes.


Currently, is there any guide that I can follow, how the suggested testframe
work can be set to the LyX environment?

Thanks
Hashini



Hi Hashini,

I'm not sure what you're asking for.

If you want to write a test using the framework I set up, you can merge 
my branch.


git remote add vfr g...@git.lyx.org:developers/vfr/lyx.git,

 and then write tests like the one I did as an example: t0001_Buffer.cpp.

Hope this helps.

Vincent

Re: LyX unit testing

[Vincent van Ravesteijn]

Op 19-9-2013 8:37, Hashini Senaratne schreef:

Hello Vincent,


But in general, I would like to introduce unit testing, yes.


Currently, is there any guide that I can follow, how the suggested testframe
work can be set to the LyX environment?

Thanks
Hashini



Hi Hashini,

I'm not sure what you're asking for.

If you want to write a test using the framework I set up, you can merge 
my branch.


git remote add vfr g...@git.lyx.org:developers/vfr/lyx.git,

 and then write tests like the one I did as an example: t0001_Buffer.cpp.

Hope this helps.

Vincent

Re: LyX unit testing

[Vincent van Ravesteijn]

Op 16-9-2013 8:16, Scott Kostyshak schreef:

Hi Vincent,

Do you have any plans to merge the gtest code
http://git.lyx.org/?p=developers/vfr/lyx.git;a=shortlog;h=refs/heads/tests
into master after the 2.1 release? Does it still need a lot of work
before it should be merged?

Hashini has expressed interest in implementing unit tests, and a
couple of questions have popped up on the list over the months so I
just wanted to get an idea of how feasible it is to have in LyX's main
repo.

Scott


Hi Scott,

I was interested in getting some feedback on whether we want to proceed 
with the google-test framework, or not ? Do we need a 3rd-party testing 
framework at all ?


Lastly, there are huge problems in implementing unit tests for existing 
pieces of code. This is because of the large number of dependencies 
between classes, the fact that a lot of functionalities are packed 
together in some large classe, etc.


But in general, I would like to introduce unit testing, yes.

Vincent

Vincent

Re: LyX unit testing

[Vincent van Ravesteijn]

Op 16-9-2013 8:16, Scott Kostyshak schreef:

Hi Vincent,

Do you have any plans to merge the gtest code
http://git.lyx.org/?p=developers/vfr/lyx.git;a=shortlog;h=refs/heads/tests
into master after the 2.1 release? Does it still need a lot of work
before it should be merged?

Hashini has expressed interest in implementing unit tests, and a
couple of questions have popped up on the list over the months so I
just wanted to get an idea of how feasible it is to have in LyX's main
repo.

Scott


Hi Scott,

I was interested in getting some feedback on whether we want to proceed 
with the google-test framework, or not ? Do we need a 3rd-party testing 
framework at all ?


Lastly, there are huge problems in implementing unit tests for existing 
pieces of code. This is because of the large number of dependencies 
between classes, the fact that a lot of functionalities are packed 
together in some large classe, etc.


But in general, I would like to introduce unit testing, yes.

Vincent

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 6-9-2013 11:59, Jean-Marc Lasgouttes schreef:

28/08/2013 22:45, Vincent van Ravesteijn:

Hi all,

I think it's time to start thinking about what is needed before we can
release beta2.

Please reply if you know about any issues so that I won't be able to
overlook them.


I would like to merge the branch features/kill-tex2lyx-define. It 
solves cleanly the problem with automake subdirs and is a long awaited 
cleanup IMO. The branch consists of small self-contained commits and 
is easy to review IMO.




I had a look at the branch, and I have the following comments:

- In the commit logs, it is nowhere mentioned why the cleanups are 
necessary.


- In commit ed55131e8, the automake subdirs is silently introduced. As 
this change has caused such a lot of work, it should really be a 
separate commit, and should explain that the other commits were needed 
to allow this. In this same commit, the TEX2LYX defines are removed. 
This should be a separate commit, or be merged with the work that was 
needed in removing the TEX2LYX defines.


- The commit about splitting the encoding stuff and moving helper 
functions into a new class, should really have some explanation why this 
is needed, and why we would need a new class for this. The construction 
around CharInfo becomes rather strange. It is defined in Encoding.cpp, 
which indicates that it should not be used outside Encoding.cpp. It is 
now used outside this file in BufferEncoding.cpp. Because of this, there 
are a lot of static functions added to the Encoding class just to allow 
BufferEncoding to use CharInfo as an incomplete type. Moreover, most of 
these functions aren't even used.  Why is this moved into a new class ? 
What was the problem that needed to be solved ? Can't we solve it with a 
private header file ?


- The file src/BufferEncoding.cpp starts with a comment that it is 
Encoding.cpp


- The class name is BufferEncodings so why don't we call the file 
BufferEncodings.cpp ? (I know Encodings in Encoding.cpp.., but is that a 
good reason?)


- The following includes in BufferEncoding.cpp are not needed:

#include BufferList.h
#include Lexer.h
#include LyXRC.h
#include support/debug.h
#include support/gettext.h
#include support/FileName.h
#include support/textutils.h
#include support/unicode.h
#include boost/cstdint.hpp
#include sstream

- The change to src/tex2lyx/tex2lyx.cpp is unexpected in a commit that 
says it removes something from ModuleList.cpp.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

On Tue, Sep 10, 2013 at 11:44 AM, Jean-Marc Lasgouttes
lasgout...@lyx.orgwrote:

 10/09/2013 09:01, Vincent van Ravesteijn:

  I had a look at the branch, and I have the following comments:

 - In the commit logs, it is nowhere mentioned why the cleanups are
 necessary.



 Indeed... But removing #ifdef's is always good, isn't it?



I don't say it ain't good, I just would like to have an insight in what
trouble the evil-#ifdefs caused... but I found it in on lyx-devel...



  - In commit ed55131e8, the automake subdirs is silently introduced. As
 this change has caused such a lot of work, it should really be a
 separate commit, and should explain that the other commits were needed
 to allow this. In this same commit, the TEX2LYX defines are removed.
 This should be a separate commit, or be merged with the work that was
 needed in removing the TEX2LYX defines.


 Yes, this should be the final commit. How can one split a commit in two?


As the commits are already published and Stephan might also be working on
them, it is safest not to change history now.  If you want to do it anyway:

git checkout features/kill-tex2lyx-defines
git rebase -i master
Mark the commit to be edited.
When the rebase stops:
git reset HEAD~1
Now, git add and git commit several times until all changes are
committed again.
git rebase --continue



  - The change to src/tex2lyx/tex2lyx.cpp is unexpected in a commit that
 says it removes something from ModuleList.cpp.


 Most of my commits are build like that: a dummy implementation is created
 to avoid an #ifdef in the code that needs the said function.


I understand the idea, but I failed to see the relation between removing
the #ifdef's in LyXModule and adding a dummy implementation for
LaTeXFeatures::isAvailable().



 Do you want me to modify the commit logs?


I'm in no position to tell you what you'll have to do. I'll rebase the
branch when it is clear to me what Stephan exactly wanted to do with the
Encodings stuff, and I might add your previous explanations to the logs.
It's hardly any effort then.

However, if you feel like you have anything to share with the world, 

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 6-9-2013 11:59, Jean-Marc Lasgouttes schreef:

28/08/2013 22:45, Vincent van Ravesteijn:

Hi all,

I think it's time to start thinking about what is needed before we can
release beta2.

Please reply if you know about any issues so that I won't be able to
overlook them.


I would like to merge the branch features/kill-tex2lyx-define. It 
solves cleanly the problem with automake subdirs and is a long awaited 
cleanup IMO. The branch consists of small self-contained commits and 
is easy to review IMO.




I had a look at the branch, and I have the following comments:

- In the commit logs, it is nowhere mentioned why the cleanups are 
necessary.


- In commit ed55131e8, the "automake subdirs" is silently introduced. As 
this change has caused such a lot of work, it should really be a 
separate commit, and should explain that the other commits were needed 
to allow this. In this same commit, the TEX2LYX defines are removed. 
This should be a separate commit, or be merged with the work that was 
needed in removing the TEX2LYX defines.


- The commit about splitting the encoding stuff and moving helper 
functions into a new class, should really have some explanation why this 
is needed, and why we would need a new class for this. The construction 
around CharInfo becomes rather strange. It is defined in Encoding.cpp, 
which indicates that it should not be used outside Encoding.cpp. It is 
now used outside this file in BufferEncoding.cpp. Because of this, there 
are a lot of static functions added to the Encoding class just to allow 
BufferEncoding to use CharInfo as an incomplete type. Moreover, most of 
these functions aren't even used.  Why is this moved into a new class ? 
What was the problem that needed to be solved ? Can't we solve it with a 
private header file ?


- The file src/BufferEncoding.cpp starts with a comment that it is 
Encoding.cpp


- The class name is BufferEncodings so why don't we call the file 
BufferEncodings.cpp ? (I know Encodings in Encoding.cpp.., but is that a 
good reason?)


- The following includes in BufferEncoding.cpp are not needed:

#include "BufferList.h"
#include "Lexer.h"
#include "LyXRC.h"
#include "support/debug.h"
#include "support/gettext.h"
#include "support/FileName.h"
#include "support/textutils.h"
#include "support/unicode.h"
#include 
#include 

- The change to src/tex2lyx/tex2lyx.cpp is unexpected in a commit that 
says it removes something from ModuleList.cpp.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

On Tue, Sep 10, 2013 at 11:44 AM, Jean-Marc Lasgouttes
<lasgout...@lyx.org>wrote:

> 10/09/2013 09:01, Vincent van Ravesteijn:
>
>  I had a look at the branch, and I have the following comments:
>>
>> - In the commit logs, it is nowhere mentioned why the cleanups are
>> necessary.
>>
>
>
> Indeed... But removing #ifdef's is always good, isn't it?
>
>

I don't say it ain't good, I just would like to have an insight in what
trouble the evil-#ifdefs caused... but I found it in on lyx-devel...


>
>  - In commit ed55131e8, the "automake subdirs" is silently introduced. As
>> this change has caused such a lot of work, it should really be a
>> separate commit, and should explain that the other commits were needed
>> to allow this. In this same commit, the TEX2LYX defines are removed.
>> This should be a separate commit, or be merged with the work that was
>> needed in removing the TEX2LYX defines.
>>
>
> Yes, this should be the final commit. How can one split a commit in two?


As the commits are already published and Stephan might also be working on
them, it is safest not to change history now.  If you want to do it anyway:

git checkout features/kill-tex2lyx-defines
git rebase -i master
Mark the commit to be edited.
When the rebase stops:
git reset HEAD~1
Now, "git add" and "git commit" several times until all changes are
committed again.
git rebase --continue



>  - The change to src/tex2lyx/tex2lyx.cpp is unexpected in a commit that
>> says it removes something from ModuleList.cpp.
>>
>
> Most of my commits are build like that: a dummy implementation is created
> to avoid an #ifdef in the code that needs the said function.


I understand the idea, but I failed to see the relation between removing
the #ifdef's in LyXModule and adding a dummy implementation for
LaTeXFeatures::isAvailable().



> Do you want me to modify the commit logs?
>

I'm in no position to tell you what you'll have to do. I'll rebase the
branch when it is clear to me what Stephan exactly wanted to do with the
Encodings stuff, and I might add your previous explanations to the logs.
It's hardly any effort then.

However, if you feel like you have anything to share with the world, 

Vincent

Re: Fix make distcheck

[Vincent van Ravesteijn]

Op 8-9-2013 19:40, Georg Baum schreef:

Vincent van Ravesteijn wrote:


Ok, thanks for having a look at it.

I came up with a rather basic version. I did not test the new targets with
anything else than GNU make and GNU bash, but since they are mainly for our
own use I don't consider this a problem. The main goal (not having the tests
in make checks) works on any autotools supported system anyway.

OK to go in?



Yes, please put it in.

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 9-9-2013 11:05, Jean-Marc Lasgouttes schreef:

06/09/2013 14:35, Vincent van Ravesteijn:

Are you asking me? This is too much autotools for me to say something
wise about it.

All I can do is to ask people to test. If no-one reports a problem, it
will be ok.


Well, I pushed it for maximal exposure :) I will revert as needed.

JMarc



Good.

Pavel tested it and didn't report a problem, so it must be ok.

Vincent

Re: Fix make distcheck

[Vincent van Ravesteijn]

Op 8-9-2013 19:40, Georg Baum schreef:

Vincent van Ravesteijn wrote:


Ok, thanks for having a look at it.

I came up with a rather basic version. I did not test the new targets with
anything else than GNU make and GNU bash, but since they are mainly for our
own use I don't consider this a problem. The main goal (not having the tests
in make checks) works on any autotools supported system anyway.

OK to go in?


Georg


I noticed that in src/Makefile.am there is still check_PROGRAMS: 
check_layout left. I guess this should be removed.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 9-9-2013 15:50, Jean-Marc Lasgouttes schreef:

28/08/2013 22:45, Vincent van Ravesteijn:

Hi all,

I think it's time to start thinking about what is needed before we can
release beta2.

Please reply if you know about any issues so that I won't be able to
overlook them.


What is the status of the features/kill-tex2lyx-define branch?

Can we merge it?

JMarc


I'm busy clearing up my todo-list. So I will have a look at it soon.

Vincent

Re: Fix make distcheck

[Vincent van Ravesteijn]

Op 9-9-2013 1:55, Pavel Sanda schreef:

Georg Baum wrote:

OK to go in?

I tested this patch helps here.

We are still having the problem I registered some time ago with
cleaning up the tree from new pot files:


It works for me.

Vincent

Re: Fix make distcheck

[Vincent van Ravesteijn]

Op 9-9-2013 21:47, Pavel Sanda schreef:

Vincent van Ravesteijn wrote:

Op 9-9-2013 1:55, Pavel Sanda schreef:

Georg Baum wrote:

OK to go in?

I tested this patch helps here.

We are still having the problem I registered some time ago with
cleaning up the tree from new pot files:

It works for me.

Yep, JMarc committed fix some hours back.Pavel
Yes, I meant that the patch from Georg in combination with JMarc's 
commit fixed all the make distcheck problems also for me.


Vincent

Re: Fix "make distcheck"

[Vincent van Ravesteijn]

Op 8-9-2013 19:40, Georg Baum schreef:

Vincent van Ravesteijn wrote:


Ok, thanks for having a look at it.

I came up with a rather basic version. I did not test the new targets with
anything else than GNU make and GNU bash, but since they are mainly for our
own use I don't consider this a problem. The main goal (not having the tests
in make checks) works on any autotools supported system anyway.

OK to go in?



Yes, please put it in.

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 9-9-2013 11:05, Jean-Marc Lasgouttes schreef:

06/09/2013 14:35, Vincent van Ravesteijn:

Are you asking me? This is too much autotools for me to say something
wise about it.

All I can do is to ask people to test. If no-one reports a problem, it
will be ok.


Well, I pushed it for maximal exposure :) I will revert as needed.

JMarc



Good.

Pavel tested it and didn't report a problem, so it must be ok.

Vincent

Re: Fix "make distcheck"

[Vincent van Ravesteijn]

Op 8-9-2013 19:40, Georg Baum schreef:

Vincent van Ravesteijn wrote:


Ok, thanks for having a look at it.

I came up with a rather basic version. I did not test the new targets with
anything else than GNU make and GNU bash, but since they are mainly for our
own use I don't consider this a problem. The main goal (not having the tests
in make checks) works on any autotools supported system anyway.

OK to go in?


Georg


I noticed that in src/Makefile.am there is still "check_PROGRAMS: 
check_layout" left. I guess this should be removed.


Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Op 9-9-2013 15:50, Jean-Marc Lasgouttes schreef:

28/08/2013 22:45, Vincent van Ravesteijn:

Hi all,

I think it's time to start thinking about what is needed before we can
release beta2.

Please reply if you know about any issues so that I won't be able to
overlook them.


What is the status of the features/kill-tex2lyx-define branch?

Can we merge it?

JMarc


I'm busy clearing up my todo-list. So I will have a look at it soon.

Vincent

Re: Fix "make distcheck"

[Vincent van Ravesteijn]

Op 9-9-2013 1:55, Pavel Sanda schreef:

Georg Baum wrote:

OK to go in?

I tested this patch helps here.

We are still having the problem I registered some time ago with
cleaning up the tree from new pot files:


It works for me.

Vincent

Re: Fix "make distcheck"

[Vincent van Ravesteijn]

Op 9-9-2013 21:47, Pavel Sanda schreef:

Vincent van Ravesteijn wrote:

Op 9-9-2013 1:55, Pavel Sanda schreef:

Georg Baum wrote:

OK to go in?

I tested this patch helps here.

We are still having the problem I registered some time ago with
cleaning up the tree from new pot files:

It works for me.

Yep, JMarc committed fix some hours back.Pavel
Yes, I meant that the patch from Georg in combination with JMarc's 
commit fixed all the make distcheck problems also for me.


Vincent

Re: back from vacation

[Vincent van Ravesteijn]

Op 6-9-2013 22:50, Uwe Stöhr schreef:

Dear clooeagues,

sorry for my silence. I am now back but will not have much time until 
the German election September 22.

I try to commit the docs stuff I received and also the po-files.

Anything I need to know, deadlines etc.?


I'm now wrapping up to prepare for 2.1beta2. So, you can expect this soon.

Vincent

Re: back from vacation

[Vincent van Ravesteijn]

Op 6-9-2013 22:50, Uwe Stöhr schreef:

Dear clooeagues,

sorry for my silence. I am now back but will not have much time until 
the German election September 22.

I try to commit the docs stuff I received and also the po-files.

Anything I need to know, deadlines etc.?


I'm now wrapping up to prepare for 2.1beta2. So, you can expect this soon.

Vincent

Re: back from vacation

[Vincent van Ravesteijn]

Sounds good. Should we switch the default UI language to de_DE before beta 2 is 
out then?

Pavel


Sorry, I don't like this kind of last-minute changes.

Can you postpone this patch till after beta 2?

Vincent

Re: back from vacation

[Vincent van Ravesteijn]

Sounds good. Should we switch the default UI language to de_DE before beta 2 is 
out then?

Pavel


Sorry, I don't like this kind of last-minute changes.

Can you postpone this patch till after beta 2?

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

On Fri, Sep 6, 2013 at 11:16 AM, Jean-Marc Lasgouttes lasgout...@lyx.orgwrote:

 06/09/2013 07:53, Pavel Sanda:

  Vincent van Ravesteijn wrote:

 Please reply if you know about any issues so that I won't be able to
 overlook them.


 #8783


 Here is a patch for this one. OK?

 JMarc



Are you asking me? This is too much autotools for me to say something wise
about it.

All I can do is to ask people to test. If no-one reports a problem, it will
be ok.

Vincent

Re: LyX + Qt5 + CMake + XCode - attached files (1/1)

[Vincent van Ravesteijn]

 Is this for branch 2.0.x or for master? Indeed it seems that the Qt5
 porting missed the Q_WS_xx defines, that need to be reevaluated one by one.
 Vincent, what is your take on this?


I indeed read that the Q_WS_* defines were replaced, but at the same time,
it still seemed to work so I assumed that magic helped us here.

I will investigate further.

Vincent

Re: Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

On Fri, Sep 6, 2013 at 11:16 AM, Jean-Marc Lasgouttes <lasgout...@lyx.org>wrote:

> 06/09/2013 07:53, Pavel Sanda:
>
>  Vincent van Ravesteijn wrote:
>>
>>> Please reply if you know about any issues so that I won't be able to
>>> overlook them.
>>>
>>
>> #8783
>>
>
> Here is a patch for this one. OK?
>
> JMarc
>


Are you asking me? This is too much autotools for me to say something wise
about it.

All I can do is to ask people to test. If no-one reports a problem, it will
be ok.

Vincent

Re: LyX + Qt5 + CMake + XCode - attached files (1/1)

[Vincent van Ravesteijn]

>
> Is this for branch 2.0.x or for master? Indeed it seems that the Qt5
> porting missed the Q_WS_xx defines, that need to be reevaluated one by one.
> Vincent, what is your take on this?
>
>
I indeed read that the Q_WS_* defines were replaced, but at the same time,
it still seemed to work so I assumed that magic helped us here.

I will investigate further.

Vincent

Re: Fix make distcheck

[Vincent van Ravesteijn]

Op 3-9-2013 21:12, Georg Baum schreef:

Vincent van Ravesteijn wrote:


Is there any volunteer that wants to fix the Autotools distcheck target
?

This basically comes down to stripping the new tests from the check
target and to introduce a different target to run the tests.

I'm not that experienced with Makefiles, that's why I'm asking.

I tried it yesterday, but did not get a working result. Basically, the
automake test support is too basic, and one has to re-invent the wheel. I'll
continue with the work unless somebody else has a better idea, but don't
expect anything before the weekend.


Georg



Ok, thanks for having a look at it.

Vincent

Re: Fix "make distcheck"

[Vincent van Ravesteijn]

Op 3-9-2013 21:12, Georg Baum schreef:

Vincent van Ravesteijn wrote:


Is there any volunteer that wants to fix the Autotools "distcheck" target
?

This basically comes down to stripping the new tests from the "check"
target and to introduce a different target to run the tests.

I'm not that experienced with Makefiles, that's why I'm asking.

I tried it yesterday, but did not get a working result. Basically, the
automake test support is too basic, and one has to re-invent the wheel. I'll
continue with the work unless somebody else has a better idea, but don't
expect anything before the weekend.


Georg



Ok, thanks for having a look at it.

Vincent

Re: PATCH: automatic copying and migration of elder user config directories Was: LyX user directory path on Mac contains PACKAGE

[Vincent van Ravesteijn]

On Mon, Sep 2, 2013 at 5:02 PM, Stephan Witt st.w...@gmx.net wrote:

 Am 02.09.2013 um 16:42 schrieb Jean-Marc Lasgouttes lasgout...@lyx.org:

  01/09/2013 20:06, Stephan Witt:
  What is the status on this work ? Do you still want to get it in beta
 2 ?
 
  Yes, please. I've answered already in the thread Re: Lyx2.1.0beta2
 issues ?.
 
  Here is the latest patch attached again.
 
  I have some questions, probably because I did not follow the discussion
 well enough.
 
  1/ why have this code in configure.py, which complicates the situation?
 I would put this in the LyX::queryUserLyXDir code. This does not mean that
 the copy code should not be written in python, but it could by another
 script.

 I never wrote python code before. So I started adding some code where it
 fit my needs best.
 And initially I tried to not touch the LyX C++ code at all.

  2/ I still do not understand why it is needed to have a comple-time
 switch. The behaviour could just be
   * if the userdir exists, use it
   * if it does not, create one and offer to import configuration from an
 existing userdir.
  I do not see what is so horrible about that, especially for people who
 do not change their userdir.

 It was horrible for the developers who are afraid the user directory
 contents gets copied if they run
 LyX with their version suffix and are used to start with an empty user
 directory.

  To me this issue has nothing to do with platform or
 --with-version-suffix, it is just a matter of offering to populate a
 non-existing userdir.



This code apparently only works when version suffix is set to -2.0 and
-1.6. In some sense, this is rather arbitrary and the use might have any
suffix he wants. On Windows the user dir looks like: LyX2.0.

Vincent

Fix make distcheck

[Vincent van Ravesteijn]

Is there any volunteer that wants to fix the Autotools distcheck target ?

This basically comes down to stripping the new tests from the check
target and to introduce a different target to run the tests.

I'm not that experienced with Makefiles, that's why I'm asking.

Vincent

Re: PATCH: automatic copying and migration of elder user config directories Was: LyX user directory path on Mac contains PACKAGE

[Vincent van Ravesteijn]

On Mon, Sep 2, 2013 at 5:02 PM, Stephan Witt  wrote:

> Am 02.09.2013 um 16:42 schrieb Jean-Marc Lasgouttes :
>
> > 01/09/2013 20:06, Stephan Witt:
> >>> What is the status on this work ? Do you still want to get it in beta
> 2 ?
> >>
> >> Yes, please. I've answered already in the thread "Re: Lyx2.1.0beta2
> issues ?".
> >>
> >> Here is the latest patch attached again.
> >
> > I have some questions, probably because I did not follow the discussion
> well enough.
> >
> > 1/ why have this code in configure.py, which complicates the situation?
> I would put this in the LyX::queryUserLyXDir code. This does not mean that
> the copy code should not be written in python, but it could by another
> script.
>
> I never wrote python code before. So I started adding some code where it
> fit my needs best.
> And initially I tried to not touch the LyX C++ code at all.
>
> > 2/ I still do not understand why it is needed to have a comple-time
> switch. The behaviour could just be
> >  * if the userdir exists, use it
> >  * if it does not, create one and offer to import configuration from an
> existing userdir.
> > I do not see what is so horrible about that, especially for people who
> do not change their userdir.
>
> It was horrible for the developers who are afraid the user directory
> contents gets copied if they run
> LyX with their version suffix and are used to start with an empty user
> directory.
>
> > To me this issue has nothing to do with platform or
> --with-version-suffix, it is just a matter of offering to populate a
> non-existing userdir.
>
>
>
This code apparently only works when version suffix is set to -2.0 and
-1.6. In some sense, this is rather arbitrary and the use might have any
suffix he wants. On Windows the user dir looks like: "LyX2.0".

Vincent

Fix "make distcheck"

[Vincent van Ravesteijn]

Is there any volunteer that wants to fix the Autotools "distcheck" target ?

This basically comes down to stripping the new tests from the "check"
target and to introduce a different target to run the tests.

I'm not that experienced with Makefiles, that's why I'm asking.

Vincent

Re: getting bibtex-database-name via LFUN

[Vincent van Ravesteijn]

Op 31-8-2013 1:42, newslet...@semiotikon.de schreef:

Hi,

i am currently working on a bibliography management tool based on 
XULRunner. Right now i'm about to implement the pushing of citations 
to lyx.


I have read the code of Petr Simon the first developer of lyz. What he 
did to manage bibtex-databases in relation with lyx is quite 
complicated: He stores the name of the lyx file and the name of the 
bibtex file in the sqlite-storage of his app, just to know into which 
bibtext file additional entries pushed to lyx should be stored.


I'm sure he would have solved it otherwise if it was possible three 
years ago. So my question is: Is it now possible to get the name of 
the bibtex file(s) attached to a lxy document via LFUN?


No, that is not possible. You can parse the LyX file, and get the 
filename. The format of the lyx file is not very complicated.


Vincent

Re: cmake build: man pages

[Vincent van Ravesteijn]

Op 25-8-2013 13:31, Kornel Benko schreef:


Am Dienstag, 20. August 2013 um 15:12:48, schrieb Kornel Benko 
kor...@lyx.org


 Am Freitag, 16. August 2013 um 10:04:15, schrieb Pavel Sanda 
sa...@lyx.org


  Kornel Benko wrote:

 This can be overridden by call to cmake. It's a default only.

   

I see it's even default for autotools, let it be this way.

   

Pavel

  

   Thanks. If nobody else screams, I will commit tomorrow.

 

  That was not commit approval, it's Vincent's business to decide 
such things now.


 

 



If there are concerns about stabilitiy, and if it doesn't fix a bug, I 
will need to ask you to postpone.


Vincent

Re: [PATCH] Add outline-{up,down} to alt-{up,down} binding

[Vincent van Ravesteijn]

Op 7-8-2013 9:01, Scott Kostyshak schreef:

On Fri, Jul 5, 2013 at 8:48 PM, Scott Kostyshak skost...@lyx.org wrote:

On Wed, Jun 12, 2013 at 1:25 PM, Scott Kostyshak skost...@lyx.org wrote:

On Tue, Jun 11, 2013 at 10:16 AM, Richard Heck rgh...@lyx.org wrote:

On 06/11/2013 12:21 AM, Pavel Sanda wrote:

Scott Kostyshak wrote:

Any thoughts?

Looks reasonable.


I don't see any reason this can't go into 2.1. It adds behavior, not changes
it.

Vincent, should I commit to master?

Vincent, should I commit to master?

Vincent,

Should this go to master?

Scott

Yes, that's ok.

Vincent

Re: PATCH: automatic copying and migration of elder user config directories Was: LyX user directory path on Mac contains PACKAGE

[Vincent van Ravesteijn]

Op 31-7-2013 15:46, Richard Heck schreef:

On 07/30/2013 04:34 PM, Stephan Witt wrote:

Am 30.07.2013 um 15:56 schrieb Stephan Wittst.w...@gmx.net:


Am 29.07.2013 um 10:05 schrieb Jürgen Spitzmüllersp...@lyx.org:


Stephan Witt wrote:

Please no. Why not make it a parameter to lyx: -inherit-settings true,
defaulting to false.

Ok. I'll do it for Mac only then.

Hm. If the build flag is an opt-in, I do not see why Mac-only should be less
intrusive. Maybe people on other platforms want this too, and maybe people
(self-compiling) on Mac don't want it.

This would be a patch to add an --enable-inherit-user-settings configure 
switch...

BTW, is it possible for me to create a git branch and make it public?
Do I need additional privileges or other git command options only?


You can easily do this in the features branch. See
http://wiki.lyx.org/Devel/LyXGit#toc3
for info.


What is the status on this work ? Do you still want to get it in beta 2 ?

Vincent

Re: [PATCH] Re: feature request: allowing Update Master Document command in master document

[Vincent van Ravesteijn]

Op 17-7-2013 11:25, Jean-Marc Lasgouttes schreef:

Le 16/07/2013 06:48, Pavel Sanda a écrit :

Jean-Marc Lasgouttes wrote:
OK, let's keep it as it is right now. Shall I commit it somewhere 
now or

wait?


I would put it into master. P



Vincent, can I commit this somewhere?

JMarc

I put it in.

Vincent

Re: getting bibtex-database-name via LFUN

[Vincent van Ravesteijn]

Op 31-8-2013 1:42, newslet...@semiotikon.de schreef:

Hi,

i am currently working on a bibliography management tool based on 
XULRunner. Right now i'm about to implement the pushing of citations 
to lyx.


I have read the code of Petr Simon the first developer of lyz. What he 
did to manage bibtex-databases in relation with lyx is quite 
complicated: He stores the name of the lyx file and the name of the 
bibtex file in the sqlite-storage of his app, just to know into which 
bibtext file additional entries pushed to lyx should be stored.


I'm sure he would have solved it otherwise if it was possible three 
years ago. So my question is: Is it now possible to get the name of 
the bibtex file(s) attached to a lxy document via LFUN?


No, that is not possible. You can parse the LyX file, and get the 
filename. The format of the lyx file is not very complicated.


Vincent

Re: cmake build: man pages

[Vincent van Ravesteijn]

Op 25-8-2013 13:31, Kornel Benko schreef:


Am Dienstag, 20. August 2013 um 15:12:48, schrieb Kornel Benko 



> Am Freitag, 16. August 2013 um 10:04:15, schrieb Pavel Sanda 



> > Kornel Benko wrote:

> > > > > This can be overridden by call to cmake. It's a default only.

> > > >

> > > > I see it's even default for autotools, let it be this way.

> > > >

> > > > Pavel

> > >

> > > Thanks. If nobody else screams, I will commit tomorrow.

> >

> > That was not commit approval, it's Vincent's business to decide 
such things now.


> >

> >



If there are concerns about stabilitiy, and if it doesn't fix a bug, I 
will need to ask you to postpone.


Vincent

Re: [PATCH] Add outline-{up,down} to alt-{up,down} binding

[Vincent van Ravesteijn]

Op 7-8-2013 9:01, Scott Kostyshak schreef:

On Fri, Jul 5, 2013 at 8:48 PM, Scott Kostyshak  wrote:

On Wed, Jun 12, 2013 at 1:25 PM, Scott Kostyshak  wrote:

On Tue, Jun 11, 2013 at 10:16 AM, Richard Heck  wrote:

On 06/11/2013 12:21 AM, Pavel Sanda wrote:

Scott Kostyshak wrote:

Any thoughts?

Looks reasonable.


I don't see any reason this can't go into 2.1. It adds behavior, not changes
it.

Vincent, should I commit to master?

Vincent, should I commit to master?

Vincent,

Should this go to master?

Scott

Yes, that's ok.

Vincent

Re: PATCH: automatic copying and migration of elder user config directories Was: LyX user directory path on Mac contains PACKAGE

[Vincent van Ravesteijn]

Op 31-7-2013 15:46, Richard Heck schreef:

On 07/30/2013 04:34 PM, Stephan Witt wrote:

Am 30.07.2013 um 15:56 schrieb Stephan Witt:


Am 29.07.2013 um 10:05 schrieb Jürgen Spitzmüller:


Stephan Witt wrote:

Please no. Why not make it a parameter to lyx: "-inherit-settings true",
defaulting to false.

Ok. I'll do it for Mac only then.

Hm. If the build flag is an opt-in, I do not see why "Mac-only" should be less
intrusive. Maybe people on other platforms want this too, and maybe people
(self-compiling) on Mac don't want it.

This would be a patch to add an --enable-inherit-user-settings configure 
switch...

BTW, is it possible for me to create a git branch and make it public?
Do I need additional privileges or other git command options only?


You can easily do this in the features branch. See
http://wiki.lyx.org/Devel/LyXGit#toc3
for info.


What is the status on this work ? Do you still want to get it in beta 2 ?

Vincent

Re: [PATCH] Re: feature request: allowing "Update Master Document" command in master document

[Vincent van Ravesteijn]

Op 17-7-2013 11:25, Jean-Marc Lasgouttes schreef:

Le 16/07/2013 06:48, Pavel Sanda a écrit :

Jean-Marc Lasgouttes wrote:
OK, let's keep it as it is right now. Shall I commit it somewhere 
now or

wait?


I would put it into master. P



Vincent, can I commit this somewhere?

JMarc

I put it in.

Vincent

Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Hi all,

I think it's time to start thinking about what is needed before we can
release beta2.

Please reply if you know about any issues so that I won't be able to
overlook them.

Vincentl

Lyx2.1.0beta2 issues ?

[Vincent van Ravesteijn]

Hi all,

I think it's time to start thinking about what is needed before we can
release beta2.

Please reply if you know about any issues so that I won't be able to
overlook them.

Vincentl

Re: Test

[Vincent van Ravesteijn]

Re: Test

[Vincent van Ravesteijn]

Re: role of export tests in 'make test'

[Vincent van Ravesteijn]

Op 7-8-2013 9:01, Scott Kostyshak schreef:

Should 'make test' run export tests at all? If so, which LaTeX

I think it shouldn't. If you insist on them they should be skipped not
err if tex class is missing.

I don't have a preference here. You, Vincent, and Kornel should decide
and I will read some references to learn common practices (and common
sense) for testing.



I'm not an expert on autotools, but I have the feeling that make check 
should only validate whether the target platform is suitable for LyX to 
run on. This means it is testing the platform, rather than testing LyX.


I've come to the conclusion that export tests should NOT be in make check.

By the way, someone needs to refurbish autotools to extract the tests 
from make check before beta2.


Vincent

Re: role of export tests in 'make test'

[Vincent van Ravesteijn]

Op 7-8-2013 13:24, Kornel Benko schreef:


 I've come to the conclusion that export tests should NOT be in make 
check.


+1, but

I could not find the place where 'make check' does export?



I was just answering Scott's question whether export tests should be or 
should not be in make check.



 By the way, someone needs to refurbish autotools to extract the tests

 from make check before beta2.

??

What is the use of 'make check' then?



To check whether the target platform is suitable to receive LyX, and 
thus should pass some tests that guarantee this.


Vincent

Re: role of export tests in 'make test'

[Vincent van Ravesteijn]

Op 7-8-2013 9:01, Scott Kostyshak schreef:

Should 'make test' run export tests at all? If so, which LaTeX

I think it shouldn't. If you insist on them they should be skipped not
err if tex class is missing.

I don't have a preference here. You, Vincent, and Kornel should decide
and I will read some references to learn common practices (and common
sense) for testing.



I'm not an expert on autotools, but I have the feeling that "make check" 
should only validate whether the target platform is suitable for LyX to 
run on. This means it is testing the platform, rather than testing LyX.


I've come to the conclusion that export tests should NOT be in make check.

By the way, someone needs to refurbish autotools to extract the tests 
from make check before beta2.


Vincent

Re: role of export tests in 'make test'

[Vincent van Ravesteijn]

Op 7-8-2013 13:24, Kornel Benko schreef:


> I've come to the conclusion that export tests should NOT be in make 
check.


+1, but

I could not find the place where 'make check' does export?



I was just answering Scott's question whether export tests should be or 
should not be in make check.



> By the way, someone needs to refurbish autotools to extract the tests

> from make check before beta2.

??

What is the use of 'make check' then?



To check whether the target platform is suitable to receive LyX, and 
thus should pass some tests that guarantee this.


Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 4-8-2013 22:23, Scott Kostyshak schreef:

On Sat, Aug 3, 2013 at 3:54 AM, Scott Kostyshak skost...@lyx.org wrote:

On Fri, Aug 2, 2013 at 5:58 AM, Vincent van Ravesteijn v...@lyx.org wrote:

On Fri, Aug 2, 2013 at 11:45 AM, Kornel Benko kor...@lyx.org wrote:

About the patch... why do we care about malformed LyX documents ? How
can
this happen ? Are there some documents in a very old fileformat. Then we
should update those.



At least now the tests will show it.



If this is a valid argument, we can throw in a thousand of ad-hoc checks at
random places, ...

I don't consider this an ad-hoc check because I don't view it as a
test for the LyX documents (although as Kornel points out it might
have this extra benefit). I view it as a test of the assumption that
the export tests are making. The export tests depend on finding the
'default_output_format' buffer param. For the tests to work correctly,
it is assumed that the buffer param exists. This patch tests that
assumption.

Scott

Vincent, do you have more comments on this?

Scott


Well, I still didn't get an answer to my question why it is needed ?

As far as I see it, it cannot happen that there is no 
default_output_format. Only if there are lyx files with an old file 
format. In that case:


- a) we should update all lyx docs to the latest format anyway;
- b) we can just check for the fileformat to see whether it has the 
latest format;

- c) we can assume default when there is no default_output_format.

Either way, it is not important, but I just don't see the rationale 
behind this patch.


Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 5-8-2013 13:07, Scott Kostyshak schreef:

On Mon, Aug 5, 2013 at 6:02 AM, Vincent van Ravesteijn v...@lyx.org wrote:

Op 4-8-2013 22:23, Scott Kostyshak schreef:


On Sat, Aug 3, 2013 at 3:54 AM, Scott Kostyshak skost...@lyx.org wrote:

On Fri, Aug 2, 2013 at 5:58 AM, Vincent van Ravesteijn v...@lyx.org
wrote:

On Fri, Aug 2, 2013 at 11:45 AM, Kornel Benko kor...@lyx.org wrote:

About the patch... why do we care about malformed LyX documents ? How
can
this happen ? Are there some documents in a very old fileformat. Then
we
should update those.



At least now the tests will show it.



If this is a valid argument, we can throw in a thousand of ad-hoc checks
at
random places, ...

I don't consider this an ad-hoc check because I don't view it as a
test for the LyX documents (although as Kornel points out it might
have this extra benefit). I view it as a test of the assumption that
the export tests are making. The export tests depend on finding the
'default_output_format' buffer param. For the tests to work correctly,
it is assumed that the buffer param exists. This patch tests that
assumption.

Scott

Vincent, do you have more comments on this?

Scott


Well, I still didn't get an answer to my question why it is needed ?

Currently when no default_output_format is found we are assuming a
default output format of 'pdf'. I tried to improve things by focusing
on the assuming part of the previous statement, but I'm fine with
improving the 'pdf' part of the assumption, as you suggest below.


Why did you assume 'pdf' in the first place ?

Now, I see that the lyx2lyx script doesn't add the default_output_format 
parameter. That looks like a bug to me.



Only if there are lyx files with an old file format. In that case:

- a) we should update all lyx docs to the latest format anyway;
- b) we can just check for the fileformat to see whether it has the latest
format;
- c) we can assume default when there is no default_output_format.

The attached patch implements c): If no 'default_output_format' is
found, then we assume 'default'.

Is it OK?

Scott


I guess so.

Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 5-8-2013 14:37, Kornel Benko schreef:


Am Montag, 5. August 2013 um 08:22:19, schrieb Scott Kostyshak 
skost...@lyx.org


 On Mon, Aug 5, 2013 at 8:09 AM, Kornel Benko kor...@lyx.org wrote:

  Am Montag, 5. August 2013 um 08:01:08, schrieb Scott Kostyshak

  skost...@lyx.org

 

  On Mon, Aug 5, 2013 at 5:56 AM, Pavel Sanda sa...@lyx.org wrote:

 

   Scott Kostyshak wrote:

 

Sorry, I meant This is also a test for lyx2lyx, isn't it?

 

  

 

   My opinion is that we should update those documents. I agree 
that it


 

  

 

   attic is just garbage and it is somewhat absurd that test 
should fail


 

   because of some contents in there. Please create proper tests for

 

   things you want to test and do not impose some future maintenance

 

   burden like we should keep stuff in attic up-to-date.

 

 

 

  OK. Attached is a patch.

 

 

 

  Scott

 

 

 

  Why not check earlier?



 Indeed, that is better. I think you can commit that (if you agree with

 it). Both Vincent and Pavel are in favor of not testing attic.



 Scott

I do not agree. We should check attic too IMHO (or else do not provide 
it).


Therefore I prefer you to commit :)

Kornel



If you would ask me, I would veto any decision to force a check of the 
attic documents. I totally agree with Pavel that it is rather stupid to 
maintain documents that we have declared obsolete.


The problem here is that it is not clear what is being tested. You seem 
to have the idea that you're testing LyX using the docs. However, if 
they were proper tests, the documents should never change unless LyX's 
behaviour changes. This is not the case here. The docs are constantly 
updated, by developers, by translators etc. If a document got broken by 
some exotic combination of preamble changes,  change tracking, and maybe 
some asian characters, the tests suddenly tells us that LyX is broken, 
while LyX didn't change.


I don't like to have to fix the chinese documents to be able to still 
use the tests. Or to fix the preamble of the obsoleted polish math 
manual, because otherwise the tests don't pass.


Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 3-8-2013 9:54, Scott Kostyshak schreef:

On Fri, Aug 2, 2013 at 2:36 PM, Kornel Benko kor...@lyx.org wrote:

Am Freitag, 2. August 2013 um 20:06:40, schrieb Vincent van Ravesteijn
v...@lyx.org


Why should we update? This is also the test for lyx2lyx, isn't it?

No, this is not the way to test lyx2lyx.
Vincent



Sorry, I meant This is also a test for lyx2lyx, isn't it?

My opinion is that we should update those documents. I agree that it
would be nice to do something to test lyx2lyx in this way, but we
should do this for more than these files. I haven't thought about the
best way to do this. In 2.1dev we could include a directory in
development/autotests/ with all of the docs and examples from the
2.0.x branch. This shouldn't increase the size of the git repository
but it would increase the size of the tar ball.


Please don't do this. Things will become a mess, and a nightmare to 
maintain.


Please write proper lyx2lyx tests if you want to test lyx2lyx.

Vincent

Re: [LyX/master] * cs.po

[Vincent van Ravesteijn]

Op 5 aug. 2013 22:21 schreef Pavel Sanda sa...@lyx.org het volgende:

 Pavel Sanda wrote:
  -POT-Creation-Date: 2013-08-05 13:08-0700\n
  +POT-Creation-Date: 2013-08-05 13:11-0700\n

 BTW Vincent, what version of gettext you use?
 Got lot of merge conflicts last time.
 Pavel

Default of Ubuntu 12.10.

Vincent

Re: [PATCH] Fix Viewing of XHTML with Math as Images

[Vincent van Ravesteijn]

Op 5 aug. 2013 23:25 schreef Richard Heck rgh...@lyx.org het volgende:

 On 08/04/2013 05:43 PM, Richard Heck wrote:


 The attached patch explains the problem and offers a simple solution.
This is master-only, and it needs to go into 2.1. OK?


 Someone want to approve this? or not?

 Richard



First, please shorten the subject line to around 60 chars.

Second, it is ugly that PreviewImage depends on Buffer.

Third, it doesn't feel good to leave behind preview images, just in case
the export XHTML needs them.

Fourth, I don't like removing files in a destructor in the first place. It
should be a conscious decision to do that. Compare also hideDialogs calls
in Inset destructors. If you would need to delete the files explicitly, it
would also be easier to not delete them when appropriate.

Fifth, do you use isClone to detect whether we are exporting ? Seems like a
hack that deserves a comment.

But you'll probably want a solution to the problem, but that's difficult to
come up with right away.

Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 4-8-2013 22:23, Scott Kostyshak schreef:

On Sat, Aug 3, 2013 at 3:54 AM, Scott Kostyshak <skost...@lyx.org> wrote:

On Fri, Aug 2, 2013 at 5:58 AM, Vincent van Ravesteijn <v...@lyx.org> wrote:

On Fri, Aug 2, 2013 at 11:45 AM, Kornel Benko <kor...@lyx.org> wrote:

About the patch... why do we care about malformed LyX documents ? How
can
this happen ? Are there some documents in a very old fileformat. Then we
should update those.



At least now the tests will show it.



If this is a valid argument, we can throw in a thousand of ad-hoc checks at
random places, ...

I don't consider this an ad-hoc check because I don't view it as a
test for the LyX documents (although as Kornel points out it might
have this extra benefit). I view it as a test of the assumption that
the export tests are making. The export tests depend on finding the
'default_output_format' buffer param. For the tests to work correctly,
it is assumed that the buffer param exists. This patch tests that
assumption.

Scott

Vincent, do you have more comments on this?

Scott


Well, I still didn't get an answer to my question why it is needed ?

As far as I see it, it cannot happen that there is no 
default_output_format. Only if there are lyx files with an old file 
format. In that case:


- a) we should update all lyx docs to the latest format anyway;
- b) we can just check for the fileformat to see whether it has the 
latest format;

- c) we can assume "default" when there is no "default_output_format".

Either way, it is not important, but I just don't see the rationale 
behind this patch.


Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 5-8-2013 13:07, Scott Kostyshak schreef:

On Mon, Aug 5, 2013 at 6:02 AM, Vincent van Ravesteijn <v...@lyx.org> wrote:

Op 4-8-2013 22:23, Scott Kostyshak schreef:


On Sat, Aug 3, 2013 at 3:54 AM, Scott Kostyshak <skost...@lyx.org> wrote:

On Fri, Aug 2, 2013 at 5:58 AM, Vincent van Ravesteijn <v...@lyx.org>
wrote:

On Fri, Aug 2, 2013 at 11:45 AM, Kornel Benko <kor...@lyx.org> wrote:

About the patch... why do we care about malformed LyX documents ? How
can
this happen ? Are there some documents in a very old fileformat. Then
we
should update those.



At least now the tests will show it.



If this is a valid argument, we can throw in a thousand of ad-hoc checks
at
random places, ...

I don't consider this an ad-hoc check because I don't view it as a
test for the LyX documents (although as Kornel points out it might
have this extra benefit). I view it as a test of the assumption that
the export tests are making. The export tests depend on finding the
'default_output_format' buffer param. For the tests to work correctly,
it is assumed that the buffer param exists. This patch tests that
assumption.

Scott

Vincent, do you have more comments on this?

Scott


Well, I still didn't get an answer to my question why it is needed ?

Currently when no default_output_format is found we are assuming a
default output format of 'pdf'. I tried to improve things by focusing
on the "assuming" part of the previous statement, but I'm fine with
improving the 'pdf' part of the assumption, as you suggest below.


Why did you assume 'pdf' in the first place ?

Now, I see that the lyx2lyx script doesn't add the default_output_format 
parameter. That looks like a bug to me.



Only if there are lyx files with an old file format. In that case:

- a) we should update all lyx docs to the latest format anyway;
- b) we can just check for the fileformat to see whether it has the latest
format;
- c) we can assume "default" when there is no "default_output_format".

The attached patch implements c): If no 'default_output_format' is
found, then we assume 'default'.

Is it OK?

Scott


I guess so.

Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 5-8-2013 14:37, Kornel Benko schreef:


Am Montag, 5. August 2013 um 08:22:19, schrieb Scott Kostyshak 



> On Mon, Aug 5, 2013 at 8:09 AM, Kornel Benko  wrote:

> > Am Montag, 5. August 2013 um 08:01:08, schrieb Scott Kostyshak

> > 

> >

> >> On Mon, Aug 5, 2013 at 5:56 AM, Pavel Sanda  wrote:

> >

> >> > Scott Kostyshak wrote:

> >

> >> >> > Sorry, I meant "This is also a test for lyx2lyx, isn't it?"

> >

> >> >>

> >

> >> >> My opinion is that we should update those documents. I agree 
that it


> >

> >> >

> >

> >> > attic is just garbage and it is somewhat absurd that test 
should fail


> >

> >> > because of some contents in there. Please create proper tests for

> >

> >> > things you want to test and do not impose some future maintenance

> >

> >> > burden like "we should keep stuff in attic up-to-date".

> >

> >>

> >

> >> OK. Attached is a patch.

> >

> >>

> >

> >> Scott

> >

> >

> >

> > Why not check earlier?

>

> Indeed, that is better. I think you can commit that (if you agree with

> it). Both Vincent and Pavel are in favor of not testing attic.

>

> Scott

I do not agree. We should check attic too IMHO (or else do not provide 
it).


Therefore I prefer you to commit :)

Kornel



If you would ask me, I would veto any decision to force a check of the 
attic documents. I totally agree with Pavel that it is rather stupid to 
maintain documents that we have declared obsolete.


The problem here is that it is not clear what is being tested. You seem 
to have the idea that you're testing LyX using the docs. However, if 
they were proper tests, the documents should never change unless LyX's 
behaviour changes. This is not the case here. The docs are constantly 
updated, by developers, by translators etc. If a document got broken by 
some exotic combination of preamble changes,  change tracking, and maybe 
some asian characters, the tests suddenly tells us that LyX is broken, 
while LyX didn't change.


I don't like to have to fix the chinese documents to be able to still 
use the tests. Or to fix the preamble of the obsoleted polish math 
manual, because otherwise the tests don't pass.


Vincent

Re: [PATCH] CMake: tests now fail if no default_output_format

[Vincent van Ravesteijn]

Op 3-8-2013 9:54, Scott Kostyshak schreef:

On Fri, Aug 2, 2013 at 2:36 PM, Kornel Benko <kor...@lyx.org> wrote:

Am Freitag, 2. August 2013 um 20:06:40, schrieb Vincent van Ravesteijn
<v...@lyx.org>


Why should we update? This is also the test for lyx2lyx, isn't it?

No, this is not the way to test lyx2lyx.
Vincent



Sorry, I meant "This is also a test for lyx2lyx, isn't it?"

My opinion is that we should update those documents. I agree that it
would be nice to do something to test lyx2lyx in this way, but we
should do this for more than these files. I haven't thought about the
best way to do this. In 2.1dev we could include a directory in
development/autotests/ with all of the docs and examples from the
2.0.x branch. This shouldn't increase the size of the git repository
but it would increase the size of the tar ball.


Please don't do this. Things will become a mess, and a nightmare to 
maintain.


Please write proper lyx2lyx tests if you want to test lyx2lyx.

Vincent

Re: [LyX/master] * cs.po

[Vincent van Ravesteijn]

Op 5 aug. 2013 22:21 schreef "Pavel Sanda"  het volgende:
>
> Pavel Sanda wrote:
> > -"POT-Creation-Date: 2013-08-05 13:08-0700\n"
> > +"POT-Creation-Date: 2013-08-05 13:11-0700\n"
>
> BTW Vincent, what version of gettext you use?
> Got lot of merge conflicts last time.
> Pavel

Default of Ubuntu 12.10.

Vincent

Re: [PATCH] Fix Viewing of XHTML with Math as Images

[Vincent van Ravesteijn]

Op 5 aug. 2013 23:25 schreef "Richard Heck"  het volgende:
>
> On 08/04/2013 05:43 PM, Richard Heck wrote:
>>
>>
>> The attached patch explains the problem and offers a simple solution.
This is master-only, and it needs to go into 2.1. OK?
>
>
> Someone want to approve this? or not?
>
>> Richard
>>
>

First, please shorten the subject line to around 60 chars.

Second, it is ugly that PreviewImage depends on Buffer.

Third, it doesn't feel good to leave behind preview images, just in case
the export XHTML needs them.

Fourth, I don't like removing files in a destructor in the first place. It
should be a conscious decision to do that. Compare also hideDialogs calls
in Inset destructors. If you would need to delete the files explicitly, it
would also be easier to not delete them when appropriate.

Fifth, do you use isClone to detect whether we are exporting ? Seems like a
hack that deserves a comment.

But you'll probably want a solution to the problem, but that's difficult to
come up with right away.

Vincent

Re: [LyX/master] tex2lyx roundtrip: expand diff to bu usefull to files created by lyx

[Vincent van Ravesteijn]

Op 4-8-2013 16:20, Kornel Benko schreef:

commit d3245d4cd20b54701af2465a825183cd105b1f56
Author: Kornel Benko kor...@lyx.org
Date:   Sun Aug 4 15:57:01 2013 +0200

 tex2lyx roundtrip: expand diff to bu usefull to files created by lyx
 
 Otherwise the comparision used by tex2lyx2lyx-tests would fail


diff --git a/src/tex2lyx/test/runtests.py b/src/tex2lyx/test/runtests.py
index 57cd0e1..5a279de 100755
--- a/src/tex2lyx/test/runtests.py
+++ b/src/tex2lyx/test/runtests.py
@@ -20,14 +20,15 @@ import os, string, sys, time, difflib, filecmp, subprocess, 
re
  def usage(prog_name):
return Usage: %s [uselyx2lyx] [tex2lyx binary [[script dir] [[output dir] 
[testfile % prog_name
  
-pat_fl = re.compile(r'^#LyX file created by tex2lyx .*$')

+pat_fl1 = re.compile(r'^#LyX file created by tex2lyx .*$')
+pat_fl2 = re.compile(r'^#LyX 2\.1 created this file.*$')


Why the need to explicitly state 2.1 ? This only means extra 
maintenance burden.


  
  def compareLyx(lines1, lines2):

  if lines1[1:] != lines2[1:]:
  return False
-if not pat_fl.match(lines1[0]):
+if not pat_fl1.match(lines1[0]) and not pat_fl2.match(lines1[0]):
  return False
-if not pat_fl.match(lines2[0]):
+if not pat_fl1.match(lines2[0]) and not pat_fl2.match(lines2[0]):
  return False
  return True
  
@@ -92,6 +93,7 @@ def main(argv):

  else:
  lyxfile = os.path.join(outputdir, base + .lyx)
  cmd = '%s -roundtrip -copyfiles -f %s %s' % (tex2lyx, texfile, 
lyxfile)
+print 'Executing: ' + cmd + \n
  proc = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE, 
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  proc.wait()
  err = proc.returncode
@@ -121,7 +123,8 @@ def main(argv):
  lines2 = f2.readlines()
  f1.close()
  f2.close()
-# ignore the first line i.e., the version of lyx
+# ignore the first lone
+# e.g. the version of lyx

Re-introducing the typo ?

Vincent