Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Thu, Apr 23, 2009 at 08:41:54AM +0200, Jürgen Spitzmüller wrote: Pavel Sanda wrote: It's in the LyX/Cygwin package here: ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz i see. imho worth to add such file somewere into development/. Why not in the manuals? I documented the reverse DVI search feature in the Extended.lyx manual. -- Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Thu, Apr 23, 2009 at 08:41:54AM +0200, Jürgen Spitzmüller wrote: > Pavel Sanda wrote: > > >> It's in the LyX/Cygwin package here: > >> ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz > > > > i see. imho worth to add such file somewere into development/. > > Why not in the manuals? I documented the reverse DVI search feature in the Extended.lyx manual. -- Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On 2009-04-22, Pavel Sanda wrote:
Enrico Forestieri wrote:
i just asked if we can have (easily) something like View-DVI
(reverse) in our output formats, which will work in normal lyx
install, without any additional tweaks so xdvi works...
I think that a button Use src-specials coud be added to
Preferences-Output-LaTeX such that \usepackage[active]{srcltx}
is inserted in the preamble.
After that, you simply need to Ctrl-click in the xdvi (maybe also
kdvi) window to initiate the inverse search. In yap, it would be still
necessary to modify the preferences from within the application.
i'm not sure this is system-wide setting - there are only some
documents you want to use it and usually only in some part of editation
process. thats why i proposed view menu... or document settings?
Could we implement this as a module instead? Then it is a
document-specific setting without additional GUI element.
secondly, instead Use src-specials use something more general
and use \usepackage{pdfsync} in preamble for pdf output...
Günter
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Pavel Sanda wrote:
It's in the LyX/Cygwin package here:
ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz
i see. imho worth to add such file somewere into development/.
Why not in the manuals?
I think that a button Use src-specials coud be added to
Preferences-Output-LaTeX such that \usepackage[active]{srcltx}
is inserted in the preamble. After that, you simply need to Ctrl-click
in the xdvi (maybe also kdvi) window to initiate the inverse search.
In yap, it would be still necessary to modify the preferences from
within the application.
i'm not sure this is system-wide setting - there are only some documents
you want to use it and usually only in some part of editation process.
thats why i proposed view menu... or document settings?
I think this is another case where we could have an rc (in LaTeX Output)
plus a document setting (in trunk: DocumentSettingsOutput). The former
could be implemented in branch, the latter only in trunk.
secondly instead instead Use src-specials use something more general
and use \usepackage{pdfsync} in preamble for pdf output...
Yes. Activate Reverse Search (however, the pdf reverse search is currently
only implemented in very few PDF viewers, at least the newer pdfsync
approach).
Jürgen
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On 2009-04-22, Pavel Sanda wrote:
> Enrico Forestieri wrote:
>> > i just asked if we can have (easily) something like View->DVI
>> > (reverse) in our output formats, which will work in normal lyx
>> > install, without any additional tweaks so xdvi works...
>> I think that a button "Use src-specials" coud be added to
>> Preferences->Output->LaTeX such that \usepackage[active]{srcltx}
>> is inserted in the preamble.
>> After that, you simply need to Ctrl-click in the xdvi (maybe also
>> kdvi) window to initiate the inverse search. In yap, it would be still
>> necessary to modify the preferences from within the application.
> i'm not sure this is system-wide setting - there are only some
> documents you want to use it and usually only in some part of editation
> process. thats why i proposed view menu... or document settings?
Could we implement this as a module instead? Then it is a
document-specific setting without additional GUI element.
> secondly, instead "Use src-specials" use something more general
> and use \usepackage{pdfsync} in preamble for pdf output...
Günter
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Pavel Sanda wrote:
>> It's in the LyX/Cygwin package here:
>> ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz
>
> i see. imho worth to add such file somewere into development/.
Why not in the manuals?
>> I think that a button "Use src-specials" coud be added to
>> Preferences->Output->LaTeX such that \usepackage[active]{srcltx}
>> is inserted in the preamble. After that, you simply need to Ctrl-click
>> in the xdvi (maybe also kdvi) window to initiate the inverse search.
>> In yap, it would be still necessary to modify the preferences from
>> within the application.
>
> i'm not sure this is system-wide setting - there are only some documents
> you want to use it and usually only in some part of editation process.
> thats why i proposed view menu... or document settings?
I think this is another case where we could have an rc (in LaTeX > Output)
plus a document setting (in trunk: Document>Settings>Output). The former
could be implemented in branch, the latter only in trunk.
> secondly instead instead "Use src-specials" use something more general
> and use \usepackage{pdfsync} in preamble for pdf output...
Yes. "Activate Reverse Search" (however, the pdf reverse search is currently
only implemented in very few PDF viewers, at least the newer pdfsync
approach).
Jürgen
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse searches from Yap back to the actual line in LyX. Is this possible? This is possible only with the cygwin version of LyX. -- Enrico
Re: LyX 1.6.2 -- svn on Windows
sykes wrote: hi, I'm running LyX 1.6.2 on Windows XP. I use the Tortoise SVN client for subversion. I'm getting an error when I try to commit from within LyX: Some problem occured while running the command: 'svn commit -m ... I'm assuming I need a command line client for svn to execute... is there a way to make Tortoise SVN handle this...or is there another application that will do it...or do I need to install cygwin and run LyX through it? CollabNet has a Windows command line client at http://www.collab.net/downloads/subversion/. You'll want it on your system command path. /Paul
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse searches from Yap back to the actual line in LyX. Is this possible? This is possible only with the cygwin version of LyX. is this version patched somehow? pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Wed, Apr 22, 2009 at 06:08:12PM +0200, Pavel Sanda wrote: Enrico Forestieri wrote: On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse searches from Yap back to the actual line in LyX. Is this possible? This is possible only with the cygwin version of LyX. is this version patched somehow? No, it works OOTB on cygwin. -- Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: On Wed, Apr 22, 2009 at 06:08:12PM +0200, Pavel Sanda wrote: Enrico Forestieri wrote: On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse searches from Yap back to the actual line in LyX. Is this possible? This is possible only with the cygwin version of LyX. is this version patched somehow? No, it works OOTB on cygwin. hmmm i recently studied this bug: http://www.lyx.org/trac/ticket/94 and thought this is not possible OOTB. you even don't use any special converter preferences or scripts for yap? is your solution extendible for xdvi? pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Wed, Apr 22, 2009 at 11:33:18PM +0200, Pavel Sanda wrote:
Enrico Forestieri wrote:
On Wed, Apr 22, 2009 at 06:08:12PM +0200, Pavel Sanda wrote:
Enrico Forestieri wrote:
On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote:
I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse
searches from Yap back to the actual line in LyX. Is this possible?
This is possible only with the cygwin version of LyX.
is this version patched somehow?
No, it works OOTB on cygwin.
hmmm i recently studied this bug:
http://www.lyx.org/trac/ticket/94
and thought this is not possible OOTB.
I think this has always been possible OOTB with xdvi, provided that
you activate src-specials when producing the dvi.
you even don't use any
special converter preferences or scripts for yap?
Only a wrapper is necessary. I attach here an excerpt of the README
file in the LyX/Cygwin package, explaining everything.
is your solution extendible for xdvi?
What do you mean? When using xdvi you only have to be sure that
src-specials are activated and then you simply Ctrl-click in the
xdvi window to jump to the right location in the LyX window.
--
Enrico
Reverse DVI search
--
Reverse DVI search allows the cursor in LyX to automatically jump to the
point corresponding to where you Ctrl-click (if using xdvi) or double click
(if using yap) in the previewed DVI file.
This feature can be enabled as follows:
* Activate src-specials by changing the LaTeX (plain)-DraftDVI converter
in Tools-Preferences-File Handling-Converters to latex --src $$i
if you use tetex, or to latex -src-specials $$i if you use miktex.
As an alternative to redefining the converter (maybe because you use
pplatex instead of latex for producing a DraftDVI), you can insert
\usepackage[active]{srcltx} in the preamble of the LyX file.
* A program or script will be called by the dvi viewer when initiating
a reverse dvi search (xdvi uses Ctrl-click, yap uses double click).
This program should take 2 arguments, a filename and a line number, and
should pass this info to a running instance of LyX. This can be done
either using the lyxpipe (set by default to ~/.lyx/lyxpipe) or the
unix domain socket that lyx creates in its temporary directory.
A suitable script (/usr/local/bin/lyxeditor.sh) is already included in
this package for using the lyxpipe machinery. You could modify it in order
to better fit your needs, but it should already work out of the box.
Alternatively, you can use lyxclient.exe for communicating with lyx
through the socket mechanism.
* If you use xdvi, you don't need to do anything else, as lyx already
provides the necessary hooks for automatically using lyxclient.exe.
However, if for whatever reason you want to use the lyxpipe instead
of the socket for communicating with lyx, simply change the DVI
viewer in Tools-Preferences-File Handling-File formats to
xdvi -editor 'lyxeditor.sh %f %l' (without double quotes), where
lyxeditor.sh is the aforementioned script.
* If you use yap, you should set the name of the program directly in yap
through the View-Options menu. However, as yap is a native Windows
application, both lyxeditor.sh and lyxclient.exe have to be launched
through the wrapper program /usr/local/bin/lyxeditor.exe (after
installation, you will find its source in the /usr/local/share/lyx/etc
directory). After launching yap, choose the View-Options menu and select
the Inverse DVI Search tab. Click on the New... button and, in the
window which opens, enter LyX Editor (or any other name you like) in
the Name: field. Now click on the button labeled ... to open a
filedialog and navigate to the directory containing lyxeditor.exe (it
will be C:\cygwin\usr\local\bin if C:\cygwin is your root directory).
Select lyxeditor.exe and then specify the program arguments as %f %l
(without the double quotes) if you want to use the lyxpipe, or as
-g %f %l (again, without quotes) if you want to use the lyxsocket.
* If you did no mistakes, and if src-specials are activated as previously
described, whenever you Ctrl-click in xdvi, or double click in yap, the
cursor in LyX should jump to the desired location.
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: is this version patched somehow? No, it works OOTB on cygwin. hmmm i recently studied this bug: http://www.lyx.org/trac/ticket/94 and thought this is not possible OOTB. I think this has always been possible OOTB with xdvi, provided that you activate src-specials when producing the dvi. by OOTB i meant without touching anything, neither prefs for output formats nor writing some wrapper... you even don't use any special converter preferences or scripts for yap? Only a wrapper is necessary. I attach here an excerpt of the README file in the LyX/Cygwin package, explaining everything. where is this file located? i just grepped our tree without success. is your solution extendible for xdvi? What do you mean? When using xdvi you only have to be sure that src-specials are activated and then you simply Ctrl-click in the xdvi window to jump to the right location in the LyX window. i just asked if we can have (easily) something like View-DVI (reverse) in our output formats, which will work in normal lyx install, without any additional tweaks so xdvi works... pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Wed, Apr 22, 2009 at 11:33:18PM +0200, Pavel Sanda wrote: hmmm i recently studied this bug: http://www.lyx.org/trac/ticket/94 and thought this is not possible OOTB. Now that I read bug 94, I think that there's some confusion there. Inverse DVI search is already implemented in LyX. What is missing is the forward DVI search, i.e., jumping to the correct position in the dvi file starting from a given position in the LyX window. So, the bug is about inverse DVI search, which is now implemented (and thus the bug should be either closed or renamed), but the last comments are about the forward DVI search feature. -- Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Thu, Apr 23, 2009 at 12:40:15AM +0200, Pavel Sanda wrote:
Enrico Forestieri wrote:
is this version patched somehow?
No, it works OOTB on cygwin.
hmmm i recently studied this bug:
http://www.lyx.org/trac/ticket/94
and thought this is not possible OOTB.
I think this has always been possible OOTB with xdvi, provided that
you activate src-specials when producing the dvi.
by OOTB i meant without touching anything, neither prefs for output
formats nor writing some wrapper...
Ha... Ok.
you even don't use any
special converter preferences or scripts for yap?
Only a wrapper is necessary. I attach here an excerpt of the README
file in the LyX/Cygwin package, explaining everything.
where is this file located? i just grepped our tree without success.
It's in the LyX/Cygwin package here:
ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz
is your solution extendible for xdvi?
What do you mean? When using xdvi you only have to be sure that
src-specials are activated and then you simply Ctrl-click in the
xdvi window to jump to the right location in the LyX window.
i just asked if we can have (easily) something like View-DVI (reverse)
in our output formats, which will work in normal lyx install, without
any additional tweaks so xdvi works...
I think that a button Use src-specials coud be added to
Preferences-Output-LaTeX such that \usepackage[active]{srcltx}
is inserted in the preamble. After that, you simply need to Ctrl-click
in the xdvi (maybe also kdvi) window to initiate the inverse search.
In yap, it would be still necessary to modify the preferences from
within the application.
--
Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote:
you even don't use any
special converter preferences or scripts for yap?
Only a wrapper is necessary. I attach here an excerpt of the README
file in the LyX/Cygwin package, explaining everything.
where is this file located? i just grepped our tree without success.
It's in the LyX/Cygwin package here:
ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz
i see. imho worth to add such file somewere into development/.
is your solution extendible for xdvi?
What do you mean? When using xdvi you only have to be sure that
src-specials are activated and then you simply Ctrl-click in the
xdvi window to jump to the right location in the LyX window.
i just asked if we can have (easily) something like View-DVI (reverse)
in our output formats, which will work in normal lyx install, without
any additional tweaks so xdvi works...
I think that a button Use src-specials coud be added to
Preferences-Output-LaTeX such that \usepackage[active]{srcltx}
is inserted in the preamble. After that, you simply need to Ctrl-click
in the xdvi (maybe also kdvi) window to initiate the inverse search.
In yap, it would be still necessary to modify the preferences from
within the application.
i'm not sure this is system-wide setting - there are only some documents you
want
to use it and usually only in some part of editation process. thats why i
proposed view menu... or document settings?
secondly instead instead Use src-specials use something more general
and use \usepackage{pdfsync} in preamble for pdf output...
pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: So, the bug is about inverse DVI search, which is now implemented (and thus the bug should be either closed or renamed), but the last comments are about the forward DVI search feature. i renamed the bug pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: > I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse > searches from Yap back to the actual line in LyX. Is this possible? This is possible only with the cygwin version of LyX. -- Enrico
Re: LyX 1.6.2 -- svn on Windows
sykes wrote: hi, I'm running LyX 1.6.2 on Windows XP. I use the Tortoise SVN client for subversion. I'm getting an error when I try to commit from within LyX: "Some problem occured while running the command: 'svn commit -m " ... " I'm assuming I need a command line client for svn to execute... is there a way to make Tortoise SVN handle this...or is there another application that will do it...or do I need to install cygwin and run LyX through it? CollabNet has a Windows command line client at http://www.collab.net/downloads/subversion/. You'll want it on your system command path. /Paul
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: > On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: > > > I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse > > searches from Yap back to the actual line in LyX. Is this possible? > > This is possible only with the cygwin version of LyX. is this version patched somehow? pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Wed, Apr 22, 2009 at 06:08:12PM +0200, Pavel Sanda wrote: > Enrico Forestieri wrote: > > On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: > > > > > I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse > > > searches from Yap back to the actual line in LyX. Is this possible? > > > > This is possible only with the cygwin version of LyX. > > is this version patched somehow? No, it works OOTB on cygwin. -- Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: > On Wed, Apr 22, 2009 at 06:08:12PM +0200, Pavel Sanda wrote: > > > Enrico Forestieri wrote: > > > On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote: > > > > > > > I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse > > > > searches from Yap back to the actual line in LyX. Is this possible? > > > > > > This is possible only with the cygwin version of LyX. > > > > is this version patched somehow? > > No, it works OOTB on cygwin. hmmm i recently studied this bug: http://www.lyx.org/trac/ticket/94 and thought this is not possible OOTB. you even don't use any special converter preferences or scripts for yap? is your solution extendible for xdvi? pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Wed, Apr 22, 2009 at 11:33:18PM +0200, Pavel Sanda wrote:
> Enrico Forestieri wrote:
> > On Wed, Apr 22, 2009 at 06:08:12PM +0200, Pavel Sanda wrote:
> >
> > > Enrico Forestieri wrote:
> > > > On Tue, Apr 21, 2009 at 07:27:46PM -0700, sykes wrote:
> > > >
> > > > > I'm running LyX 1.6.2 on Windows XP. I would like to perform inverse
> > > > > searches from Yap back to the actual line in LyX. Is this possible?
> > > > >
> > > >
> > > > This is possible only with the cygwin version of LyX.
> > >
> > > is this version patched somehow?
> >
> > No, it works OOTB on cygwin.
>
> hmmm i recently studied this bug:
> http://www.lyx.org/trac/ticket/94
> and thought this is not possible OOTB.
I think this has always been possible OOTB with xdvi, provided that
you activate src-specials when producing the dvi.
> you even don't use any
> special converter preferences or scripts for yap?
Only a wrapper is necessary. I attach here an excerpt of the README
file in the LyX/Cygwin package, explaining everything.
> is your solution extendible for xdvi?
What do you mean? When using xdvi you only have to be sure that
src-specials are activated and then you simply Ctrl-click in the
xdvi window to jump to the right location in the LyX window.
--
Enrico
Reverse DVI search
--
Reverse DVI search allows the cursor in LyX to automatically jump to the
point corresponding to where you Ctrl-click (if using xdvi) or double click
(if using yap) in the previewed DVI file.
This feature can be enabled as follows:
* Activate src-specials by changing the "LaTeX (plain)->DraftDVI" converter
in Tools->Preferences->File Handling->Converters to "latex --src $$i"
if you use tetex, or to "latex -src-specials $$i" if you use miktex.
As an alternative to redefining the converter (maybe because you use
pplatex instead of latex for producing a DraftDVI), you can insert
"\usepackage[active]{srcltx}" in the preamble of the LyX file.
* A program or script will be called by the dvi viewer when initiating
a reverse dvi search (xdvi uses Ctrl-click, yap uses double click).
This program should take 2 arguments, a filename and a line number, and
should pass this info to a running instance of LyX. This can be done
either using the lyxpipe (set by default to ~/.lyx/lyxpipe) or the
unix domain socket that lyx creates in its temporary directory.
A suitable script (/usr/local/bin/lyxeditor.sh) is already included in
this package for using the lyxpipe machinery. You could modify it in order
to better fit your needs, but it should already work out of the box.
Alternatively, you can use lyxclient.exe for communicating with lyx
through the socket mechanism.
* If you use xdvi, you don't need to do anything else, as lyx already
provides the necessary hooks for automatically using lyxclient.exe.
However, if for whatever reason you want to use the lyxpipe instead
of the socket for communicating with lyx, simply change the DVI
viewer in "Tools->Preferences->File Handling->File formats" to
"xdvi -editor 'lyxeditor.sh %f %l'" (without double quotes), where
lyxeditor.sh is the aforementioned script.
* If you use yap, you should set the name of the program directly in yap
through the View->Options menu. However, as yap is a native Windows
application, both lyxeditor.sh and lyxclient.exe have to be launched
through the wrapper program /usr/local/bin/lyxeditor.exe (after
installation, you will find its source in the /usr/local/share/lyx/etc
directory). After launching yap, choose the View->Options menu and select
the "Inverse DVI Search" tab. Click on the "New..." button and, in the
window which opens, enter "LyX Editor" (or any other name you like) in
the "Name:" field. Now click on the button labeled "..." to open a
filedialog and navigate to the directory containing lyxeditor.exe (it
will be C:\cygwin\usr\local\bin if C:\cygwin is your root directory).
Select lyxeditor.exe and then specify the program arguments as "%f %l"
(without the double quotes) if you want to use the lyxpipe, or as
"-g %f %l" (again, without quotes) if you want to use the lyxsocket.
* If you did no mistakes, and if src-specials are activated as previously
described, whenever you Ctrl-click in xdvi, or double click in yap, the
cursor in LyX should jump to the desired location.
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: > > > > is this version patched somehow? > > > > > > No, it works OOTB on cygwin. > > > > hmmm i recently studied this bug: > > http://www.lyx.org/trac/ticket/94 > > and thought this is not possible OOTB. > > I think this has always been possible OOTB with xdvi, provided that > you activate src-specials when producing the dvi. by OOTB i meant without touching anything, neither prefs for output formats nor writing some wrapper... > > you even don't use any > > special converter preferences or scripts for yap? > > Only a wrapper is necessary. I attach here an excerpt of the README > file in the LyX/Cygwin package, explaining everything. where is this file located? i just grepped our tree without success. > > is your solution extendible for xdvi? > > What do you mean? When using xdvi you only have to be sure that > src-specials are activated and then you simply Ctrl-click in the > xdvi window to jump to the right location in the LyX window. i just asked if we can have (easily) something like View->DVI (reverse) in our output formats, which will work in normal lyx install, without any additional tweaks so xdvi works... pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Wed, Apr 22, 2009 at 11:33:18PM +0200, Pavel Sanda wrote: > hmmm i recently studied this bug: > http://www.lyx.org/trac/ticket/94 > and thought this is not possible OOTB. Now that I read bug 94, I think that there's some confusion there. Inverse DVI search is already implemented in LyX. What is missing is the "forward DVI search", i.e., jumping to the correct position in the dvi file starting from a given position in the LyX window. So, the bug is about "inverse DVI search", which is now implemented (and thus the bug should be either closed or renamed), but the last comments are about the "forward DVI search" feature. -- Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
On Thu, Apr 23, 2009 at 12:40:15AM +0200, Pavel Sanda wrote:
> Enrico Forestieri wrote:
> > > > > is this version patched somehow?
> > > >
> > > > No, it works OOTB on cygwin.
> > >
> > > hmmm i recently studied this bug:
> > > http://www.lyx.org/trac/ticket/94
> > > and thought this is not possible OOTB.
> >
> > I think this has always been possible OOTB with xdvi, provided that
> > you activate src-specials when producing the dvi.
>
> by OOTB i meant without touching anything, neither prefs for output
> formats nor writing some wrapper...
Ha... Ok.
> > > you even don't use any
> > > special converter preferences or scripts for yap?
> >
> > Only a wrapper is necessary. I attach here an excerpt of the README
> > file in the LyX/Cygwin package, explaining everything.
>
> where is this file located? i just grepped our tree without success.
It's in the LyX/Cygwin package here:
ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz
> > > is your solution extendible for xdvi?
> >
> > What do you mean? When using xdvi you only have to be sure that
> > src-specials are activated and then you simply Ctrl-click in the
> > xdvi window to jump to the right location in the LyX window.
>
> i just asked if we can have (easily) something like View->DVI (reverse)
> in our output formats, which will work in normal lyx install, without
> any additional tweaks so xdvi works...
I think that a button "Use src-specials" coud be added to
Preferences->Output->LaTeX such that \usepackage[active]{srcltx}
is inserted in the preamble. After that, you simply need to Ctrl-click
in the xdvi (maybe also kdvi) window to initiate the inverse search.
In yap, it would be still necessary to modify the preferences from
within the application.
--
Enrico
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote:
> > > > you even don't use any
> > > > special converter preferences or scripts for yap?
> > >
> > > Only a wrapper is necessary. I attach here an excerpt of the README
> > > file in the LyX/Cygwin package, explaining everything.
> >
> > where is this file located? i just grepped our tree without success.
>
> It's in the LyX/Cygwin package here:
> ftp://ftp.lyx.org/pub/lyx/bin/1.6.2/lyx-1.6.2-cygwin.tar.gz
i see. imho worth to add such file somewere into development/.
> > > > is your solution extendible for xdvi?
> > >
> > > What do you mean? When using xdvi you only have to be sure that
> > > src-specials are activated and then you simply Ctrl-click in the
> > > xdvi window to jump to the right location in the LyX window.
> >
> > i just asked if we can have (easily) something like View->DVI (reverse)
> > in our output formats, which will work in normal lyx install, without
> > any additional tweaks so xdvi works...
>
> I think that a button "Use src-specials" coud be added to
> Preferences->Output->LaTeX such that \usepackage[active]{srcltx}
> is inserted in the preamble. After that, you simply need to Ctrl-click
> in the xdvi (maybe also kdvi) window to initiate the inverse search.
> In yap, it would be still necessary to modify the preferences from
> within the application.
i'm not sure this is system-wide setting - there are only some documents you
want
to use it and usually only in some part of editation process. thats why i
proposed view menu... or document settings?
secondly instead instead "Use src-specials" use something more general
and use \usepackage{pdfsync} in preamble for pdf output...
pavel
Re: LyX 1.6.2 -- does it provide inverse search via Yap ?
Enrico Forestieri wrote: > So, the bug is about "inverse DVI search", which is now implemented (and > thus the bug should be either closed or renamed), but the last comments > are about the "forward DVI search" feature. i renamed the bug pavel
Re: LyX 1.6.2: possible performance issue inside master document
Georg Baum wrote: Abdelrazak Younes wrote: I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. Yes, this would be the safest solution. I only wonder why you did not have this idea when you changed operator== from the cheap version that my patch restores to the current, expensive one... Time... This file handling clean up sucked an awful lot of my time and I was pretty fed up with it. If you did those changes on a case by case basis everything would be fine now ;-) I know, mea culpa :-) Abdel. Georg
Re: LyX 1.6.2: possible performance issue inside master document
Georg Baum wrote: Abdelrazak Younes wrote: I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. Yes, this would be the safest solution. I only wonder why you did not have this idea when you changed operator== from the cheap version that my patch restores to the current, expensive one... Time... This file handling clean up sucked an awful lot of my time and I was pretty fed up with it. If you did those changes on a case by case basis everything would be fine now ;-) I know, mea culpa :-) Abdel. Georg
Re: LyX 1.6.2: possible performance issue inside master document
Andre Poenitz wrote: On Tue, Mar 24, 2009 at 08:02:12AM -0400, rgheck wrote: I had a similar concern. Just changing the semantics of operator== seems dangerous. But another option would be, short term, to replace it by two methods, get things working again, and then make one of them back into operator==. I have no problems with two named functions wih names describing what they do exactly instead of trying to find proper semantics for operator==() Good point. Abdel.
Re: LyX 1.6.2: possible performance issue inside master document
Abdelrazak Younes wrote: I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. Yes, this would be the safest solution. I only wonder why you did not have this idea when you changed operator== from the cheap version that my patch restores to the current, expensive one... If you did those changes on a case by case basis everything would be fine now ;-) Georg
Re: LyX 1.6.2: possible performance issue inside master document
Andre Poenitz wrote: On Tue, Mar 24, 2009 at 08:02:12AM -0400, rgheck wrote: I had a similar concern. Just changing the semantics of operator== seems dangerous. But another option would be, short term, to replace it by two methods, get things working again, and then make one of them back into operator==. I have no problems with two "named" functions wih names describing what they do exactly instead of trying to find proper semantics for operator==() Good point. Abdel.
Re: LyX 1.6.2: possible performance issue inside master document
Abdelrazak Younes wrote: > I agree that we need two comparison methods. I am not sure though that > the operator==() should be lighter one. I 'd rather look at the all the > cases where it is used and replace them on a case by case with > equivalent() or isSimilarTo() as I suggested earlier. I fear that we > will introduce some bugs if we make the light comparison the default. Yes, this would be the safest solution. I only wonder why you did not have this idea when you changed operator== from the cheap version that my patch restores to the current, expensive one... If you did those changes on a case by case basis everything would be fine now ;-) Georg
Re: LyX 1.6.2: possible performance issue inside master document
Hi Georg, Georg Baum wrote: Even if you ignore the InsetInclude problem for a moment, there will be more performance problems caused by the current FileName implementation. The reason is simple: FileName once was a lightweight wrapper around a string, and it is still used in many places like that. Of course you could introduce a cache every time you encounter a performance problem, but why is that better than separating FileName::operator==() and FileName::equivalent()? Those two operations serve two different purposes and should not be mixed IMHO. I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. Abdel.
Re: LyX 1.6.2: possible performance issue inside master document
Abdelrazak Younes wrote: Hi Georg, Georg Baum wrote: Even if you ignore the InsetInclude problem for a moment, there will be more performance problems caused by the current FileName implementation. The reason is simple: FileName once was a lightweight wrapper around a string, and it is still used in many places like that. Of course you could introduce a cache every time you encounter a performance problem, but why is that better than separating FileName::operator==() and FileName::equivalent()? Those two operations serve two different purposes and should not be mixed IMHO. I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. I had a similar concern. Just changing the semantics of operator== seems dangerous. But another option would be, short term, to replace it by two methods, get things working again, and then make one of them back into operator==. rh
Re: LyX 1.6.2: possible performance issue inside master document
On Tue, Mar 24, 2009 at 08:02:12AM -0400, rgheck wrote: I had a similar concern. Just changing the semantics of operator== seems dangerous. But another option would be, short term, to replace it by two methods, get things working again, and then make one of them back into operator==. I have no problems with two named functions wih names describing what they do exactly instead of trying to find proper semantics for operator==() Andre'
Re: LyX 1.6.2: possible performance issue inside master document
Hi Georg, Georg Baum wrote: Even if you ignore the InsetInclude problem for a moment, there will be more performance problems caused by the current FileName implementation. The reason is simple: FileName once was a lightweight wrapper around a string, and it is still used in many places like that. Of course you could introduce a cache every time you encounter a performance problem, but why is that better than separating FileName::operator==() and FileName::equivalent()? Those two operations serve two different purposes and should not be mixed IMHO. I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. Abdel.
Re: LyX 1.6.2: possible performance issue inside master document
Abdelrazak Younes wrote: Hi Georg, Georg Baum wrote: Even if you ignore the InsetInclude problem for a moment, there will be more performance problems caused by the current FileName implementation. The reason is simple: FileName once was a lightweight wrapper around a string, and it is still used in many places like that. Of course you could introduce a cache every time you encounter a performance problem, but why is that better than separating FileName::operator==() and FileName::equivalent()? Those two operations serve two different purposes and should not be mixed IMHO. I agree that we need two comparison methods. I am not sure though that the operator==() should be lighter one. I 'd rather look at the all the cases where it is used and replace them on a case by case with equivalent() or isSimilarTo() as I suggested earlier. I fear that we will introduce some bugs if we make the light comparison the default. I had a similar concern. Just changing the semantics of operator== seems dangerous. But another option would be, short term, to replace it by two methods, get things working again, and then make one of them back into operator==. rh
Re: LyX 1.6.2: possible performance issue inside master document
On Tue, Mar 24, 2009 at 08:02:12AM -0400, rgheck wrote: > I had a similar concern. Just changing the semantics of operator== seems > dangerous. But another option would be, short term, to replace it by two > methods, get things working again, and then make one of them back into > operator==. I have no problems with two "named" functions wih names describing what they do exactly instead of trying to find proper semantics for operator==() Andre'
RE: Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: Yes, that is the case I am aware of. I just tried, and the patch ensures that you cannot open two buffers in that case. Note that there is still a bug in setting path.d-name in FileName::onlyPath(), this needs to be set from path.d-fi, not d-fi. Oh, and I encounter an assertion when I try to open a file from File-Recent files while already a file A.lyx is open. Then I get an assertion that \path.to\A.lyx\ is not a directory, or is this this bug ? It is the same bug. Wouldn't it be possible to cache the absolute filename of an InsetInclude ? Constructing the absolute filename each time seems like a waste of effort. It is assumed at many places in LyX that it is cheap (string concatenation). The only thing that can happen to invalidate the cached value is that the inset points to a link and that link might point to something during the editing of a document. Is that a real possibility that someone would do that ? I guess that this scenario is sufficiently strange that it does not need to be supported. This is however not the only reason why a cached name could become invalid. A far more likely case is if I rename some base directory of my project (BTDT by accident). Cautious people would not do that either while the project is open, but it works with many programs, so why should it fail with LyX? Even if you ignore the InsetInclude problem for a moment, there will be more performance problems caused by the current FileName implementation. The reason is simple: FileName once was a lightweight wrapper around a string, and it is still used in many places like that. Of course you could introduce a cache every time you encounter a performance problem, but why is that better than separating FileName::operator==() and FileName::equivalent()? Those two operations serve two different purposes and should not be mixed IMHO. Georg
RE: Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: >>Yes, that is the case I am aware of. I just tried, and the patch > ensures >>that you cannot open two buffers in that case. Note that there is still >>a bug in setting path.d->name in FileName::onlyPath(), this needs to be >>set from path.d->fi, not d->fi. > > Oh, and I encounter an assertion when I try to open a file from > File->Recent files while already a file A.lyx is open. Then I get an > assertion that \path.to\A.lyx\ is not a directory, or is this this bug ? It is the same bug. > Wouldn't it be possible to cache the absolute filename of an > InsetInclude ? Constructing the absolute filename each time seems like a > waste of effort. It is assumed at many places in LyX that it is cheap (string concatenation). > The only thing that can happen to invalidate the cached > value is that the inset points to a link and that link might point to > something during the editing of a document. Is that a real possibility > that someone would do that ? I guess that this scenario is sufficiently strange that it does not need to be supported. This is however not the only reason why a cached name could become invalid. A far more likely case is if I rename some base directory of my project (BTDT by accident). Cautious people would not do that either while the project is open, but it works with many programs, so why should it fail with LyX? Even if you ignore the InsetInclude problem for a moment, there will be more performance problems caused by the current FileName implementation. The reason is simple: FileName once was a lightweight wrapper around a string, and it is still used in many places like that. Of course you could introduce a cache every time you encounter a performance problem, but why is that better than separating FileName::operator==() and FileName::equivalent()? Those two operations serve two different purposes and should not be mixed IMHO. Georg
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote:
Georg Baum wrote:
I believe that the performance problems could be solved quite easily by
basing FileName::operator==() on simple string comparison again and
implenting an equivalence() method that is used only when needed.
Unfortunately, the case where equivalence() is needed is one of the
cases causing the performance problems.
Honest question: Really?
The only case I am aware of where equivalence() is needed is loading of .lyx
files (please correct me if I am wrong). Under that assumption, the
attached patch should fix the performance issues in almost all cases: Until
all child documents are loaded, BufferList::getBuffer() uses the expensive
method. Once all buffers are loaded, the simple method already succeeds.
Maybe the asserts in the FileName constructors destroy the performance
again. If that is the case, one could use them only in debug mode. Note
that I could not test the patch since I don't have a slow document. It is
also possible that the new implementation of isAbsolute() causes a
performance issue as well. If that is the case it could easily be solved by
using the implementation of LyX 1.5.x. AFAIK it did work well and was only
removed because qt is nicer.
Of course the performance issue is still there if a user loads a child
document as File.lyx, but the include inset uses file.lyx. Then the
expensive method will always be used. The special case of filenames
differing only in case on a case-insensitive file system could be catched
easily as well, so the only remaining cases are those dealing with symlinks
or different mount points etc.
I don't believe that those cases can be made faster at all. Any sort of
cache you introduce (e.g. the canonical name mentioned elsewehere) can
become out of date, so if you want to be correct you must not cache the
equivalence info.
Apart from the performance changes, the patch corrects the following issues:
1) FileName is still documented as storing absolute names only, but this is
not true anymore. The patch restores the old behaviour. This may look
inconvenient, but this is on purpose: relative file names should not
blindly be converted to absolute ones, the programmer should think about
the correct base. The automatic conversion of relative names was used up to
LyX 1.2.x (I think) and caused lots of problems.
2) The enforced absolute file names discovered one wrong usage of
findtexfile() in the bibtex inset. You will see that if you open the
userguide from a different directory, e.g. lyx ../Userguide.lyx.
3) typo in GuiView
Georg
Index: src/insets/InsetInclude.cpp
===
--- src/insets/InsetInclude.cpp (Revision 28885)
+++ src/insets/InsetInclude.cpp (Arbeitskopie)
@@ -446,7 +446,7 @@ int InsetInclude::latex(odocstream os,
// if incfile is relative, make it relative to the master
// buffer directory.
- if (!FileName(incfile).isAbsolute()) {
+ if (!FileName::isAbsolute(incfile)) {
// FIXME UNICODE
incfile = to_utf8(makeRelPath(from_utf8(included_file.absFilename()),
from_utf8(masterBuffer-filePath(;
Index: src/insets/InsetBibtex.cpp
===
--- src/insets/InsetBibtex.cpp (Revision 28885)
+++ src/insets/InsetBibtex.cpp (Arbeitskopie)
@@ -219,7 +219,7 @@ static string normalizeName(Buffer const
OutputParams const runparams, string const name, string const ext)
{
string const fname = makeAbsPath(name, buffer.filePath()).absFilename();
- if (FileName(name).isAbsolute() || !FileName(fname + ext).isReadableFile())
+ if (FileName::isAbsolute(name) || !FileName(fname + ext).isReadableFile())
return name;
if (!runparams.nice)
return fname;
@@ -408,8 +408,7 @@ support::FileNameList InsetBibtex::getBi
vectordocstring::const_iterator it = bibfilelist.begin();
vectordocstring::const_iterator en = bibfilelist.end();
for (; it != en; ++it) {
- FileName const file =
- findtexfile(changeExtension(to_utf8(*it), bib), bib);
+ FileName const file = getBibTeXPath(*it, buffer());
if (!file.empty())
vec.push_back(file);
Index: src/insets/ExternalSupport.cpp
===
--- src/insets/ExternalSupport.cpp (Revision 28885)
+++ src/insets/ExternalSupport.cpp (Arbeitskopie)
@@ -130,7 +130,7 @@ string const doSubstitution(InsetExterna
relToParentPath, use_latex_path,
PROTECT_EXTENSION,
ESCAPE_DOTS);
- if (FileName(filename).isAbsolute()) {
+ if (FileName::isAbsolute(filename)) {
result = subst_path(result, $$AbsOrRelPathMaster,
abspath, use_latex_path,
PROTECT_EXTENSION,
Index: src/LaTeX.cpp
===
--- src/LaTeX.cpp (Revision 28885)
+++ src/LaTeX.cpp (Arbeitskopie)
@@ -773,12 +773,13 @@ bool handleFoundFile(string const ff,
// (1) foundfile is an
// absolute path and should
// be
Re: LyX 1.6.2: possible performance issue inside master document
Georg Baum wrote: rgheck wrote: Georg Baum wrote: I believe that the performance problems could be solved quite easily by basing FileName::operator==() on simple string comparison again and implenting an equivalence() method that is used only when needed. Unfortunately, the case where equivalence() is needed is one of the cases causing the performance problems. Honest question: Really? I could be wrong, but I think so, yes. The problem is that an absolute filename isn't the same as a canonical filename, i.e., one with symlinks etc resolved. We don't want to open /home/me/file.lyx and /tmp/file.lyx in different buffers if, in fact, they are the same file. But maybe the patch works if we store the latter, rather than the former? Then again, it's hard to check whether we have a canonical path, so I'll have to think about this. Richard
Re: LyX 1.6.2: possible performance issue inside master document
On Sun, Mar 22, 2009 at 01:48:39PM +0100, Georg Baum wrote: Apart from the performance changes, the patch corrects the following issues: I can confirm that this patch solves the slowness problem with child documents (I only checked it on Solaris). Well spotted Georg! -- Enrico
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: Georg Baum wrote: rgheck wrote: Unfortunately, the case where equivalence() is needed is one of the cases causing the performance problems. Honest question: Really? I could be wrong, but I think so, yes. The problem is that an absolute filename isn't the same as a canonical filename, i.e., one with symlinks etc resolved. We don't want to open /home/me/file.lyx and /tmp/file.lyx in different buffers if, in fact, they are the same file. Yes, that is the case I am aware of. I just tried, and the patch ensures that you cannot open two buffers in that case. Note that there is still a bug in setting path.d-name in FileName::onlyPath(), this needs to be set from path.d-fi, not d-fi. But maybe the patch works if we store the latter, rather than the former? Then again, it's hard to check whether we have a canonical path, so I'll have to think about this. I believe that you do not need to care about canonical paths at all, except when loading .lyx files. The question is whether that assumption is correct. Georg
RE: Re: LyX 1.6.2: possible performance issue inside master document
I could be wrong, but I think so, yes. The problem is that an absolute filename isn't the same as a canonical filename, i.e., one with symlinks etc resolved. We don't want to open /home/me/file.lyx and /tmp/file.lyx in different buffers if, in fact, they are the same file. Yes, that is the case I am aware of. I just tried, and the patch ensures that you cannot open two buffers in that case. Note that there is still a bug in setting path.d-name in FileName::onlyPath(), this needs to be set from path.d-fi, not d-fi. Oh, and I encounter an assertion when I try to open a file from File-Recent files while already a file A.lyx is open. Then I get an assertion that \path.to\A.lyx\ is not a directory, or is this this bug ? But maybe the patch works if we store the latter, rather than the former? Then again, it's hard to check whether we have a canonical path, so I'll have to think about this. I believe that you do not need to care about canonical paths at all, except when loading .lyx files. The question is whether that assumption is correct. Wouldn't it be possible to cache the absolute filename of an InsetInclude ? Constructing the absolute filename each time seems like a waste of effort. The only thing that can happen to invalidate the cached value is that the inset points to a link and that link might point to something during the editing of a document. Is that a real possibility that someone would do that ? Georg Vincent
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote:
> Georg Baum wrote:
>> I believe that the performance problems could be solved quite easily by
>> basing FileName::operator==() on simple string comparison again and
>> implenting an equivalence() method that is used only when needed.
>>
>>
> Unfortunately, the case where equivalence() is needed is one of the
> cases causing the performance problems.
Honest question: Really?
The only case I am aware of where equivalence() is needed is loading of .lyx
files (please correct me if I am wrong). Under that assumption, the
attached patch should fix the performance issues in almost all cases: Until
all child documents are loaded, BufferList::getBuffer() uses the expensive
method. Once all buffers are loaded, the simple method already succeeds.
Maybe the asserts in the FileName constructors destroy the performance
again. If that is the case, one could use them only in debug mode. Note
that I could not test the patch since I don't have a slow document. It is
also possible that the new implementation of isAbsolute() causes a
performance issue as well. If that is the case it could easily be solved by
using the implementation of LyX 1.5.x. AFAIK it did work well and was only
removed because "qt is nicer".
Of course the performance issue is still there if a user loads a child
document as "File.lyx", but the include inset uses "file.lyx". Then the
expensive method will always be used. The special case of filenames
differing only in case on a case-insensitive file system could be catched
easily as well, so the only remaining cases are those dealing with symlinks
or different mount points etc.
I don't believe that those cases can be made faster at all. Any sort of
cache you introduce (e.g. the canonical name mentioned elsewehere) can
become out of date, so if you want to be correct you must not cache the
equivalence info.
Apart from the performance changes, the patch corrects the following issues:
1) FileName is still documented as storing absolute names only, but this is
not true anymore. The patch restores the old behaviour. This may look
inconvenient, but this is on purpose: relative file names should not
blindly be converted to absolute ones, the programmer should think about
the correct base. The automatic conversion of relative names was used up to
LyX 1.2.x (I think) and caused lots of problems.
2) The enforced absolute file names discovered one wrong usage of
findtexfile() in the bibtex inset. You will see that if you open the
userguide from a different directory, e.g. "lyx ../Userguide.lyx".
3) typo in GuiView
Georg
Index: src/insets/InsetInclude.cpp
===
--- src/insets/InsetInclude.cpp (Revision 28885)
+++ src/insets/InsetInclude.cpp (Arbeitskopie)
@@ -446,7 +446,7 @@ int InsetInclude::latex(odocstream & os,
// if incfile is relative, make it relative to the master
// buffer directory.
- if (!FileName(incfile).isAbsolute()) {
+ if (!FileName::isAbsolute(incfile)) {
// FIXME UNICODE
incfile = to_utf8(makeRelPath(from_utf8(included_file.absFilename()),
from_utf8(masterBuffer->filePath(;
Index: src/insets/InsetBibtex.cpp
===
--- src/insets/InsetBibtex.cpp (Revision 28885)
+++ src/insets/InsetBibtex.cpp (Arbeitskopie)
@@ -219,7 +219,7 @@ static string normalizeName(Buffer const
OutputParams const & runparams, string const & name, string const & ext)
{
string const fname = makeAbsPath(name, buffer.filePath()).absFilename();
- if (FileName(name).isAbsolute() || !FileName(fname + ext).isReadableFile())
+ if (FileName::isAbsolute(name) || !FileName(fname + ext).isReadableFile())
return name;
if (!runparams.nice)
return fname;
@@ -408,8 +408,7 @@ support::FileNameList InsetBibtex::getBi
vector::const_iterator it = bibfilelist.begin();
vector::const_iterator en = bibfilelist.end();
for (; it != en; ++it) {
- FileName const file =
- findtexfile(changeExtension(to_utf8(*it), "bib"), "bib");
+ FileName const file = getBibTeXPath(*it, buffer());
if (!file.empty())
vec.push_back(file);
Index: src/insets/ExternalSupport.cpp
===
--- src/insets/ExternalSupport.cpp (Revision 28885)
+++ src/insets/ExternalSupport.cpp (Arbeitskopie)
@@ -130,7 +130,7 @@ string const doSubstitution(InsetExterna
relToParentPath, use_latex_path,
PROTECT_EXTENSION,
ESCAPE_DOTS);
- if (FileName(filename).isAbsolute()) {
+ if (FileName::isAbsolute(filename)) {
result = subst_path(result, "$$AbsOrRelPathMaster",
abspath, use_latex_path,
PROTECT_EXTENSION,
Index: src/LaTeX.cpp
===
--- src/LaTeX.cpp (Revision 28885)
+++ src/LaTeX.cpp (Arbeitskopie)
@@ -773,12 +773,13 @@ bool handleFoundFile(string const & ff,
// (1) foundfile is an
// absolute path and should
Re: LyX 1.6.2: possible performance issue inside master document
Georg Baum wrote: rgheck wrote: Georg Baum wrote: I believe that the performance problems could be solved quite easily by basing FileName::operator==() on simple string comparison again and implenting an equivalence() method that is used only when needed. Unfortunately, the case where equivalence() is needed is one of the cases causing the performance problems. Honest question: Really? I could be wrong, but I think so, yes. The problem is that an absolute filename isn't the same as a canonical filename, i.e., one with symlinks etc resolved. We don't want to open /home/me/file.lyx and /tmp/file.lyx in different buffers if, in fact, they are the same file. But maybe the patch works if we store the latter, rather than the former? Then again, it's hard to check whether we have a canonical path, so I'll have to think about this. Richard
Re: LyX 1.6.2: possible performance issue inside master document
On Sun, Mar 22, 2009 at 01:48:39PM +0100, Georg Baum wrote: > Apart from the performance changes, the patch corrects the following issues: I can confirm that this patch solves the slowness problem with child documents (I only checked it on Solaris). Well spotted Georg! -- Enrico
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: > Georg Baum wrote: >> rgheck wrote: >> >>> Unfortunately, the case where equivalence() is needed is one of the >>> cases causing the performance problems. >>> >> >> Honest question: Really? >> >> > I could be wrong, but I think so, yes. The problem is that an absolute > filename isn't the same as a canonical filename, i.e., one with symlinks > etc resolved. We don't want to open /home/me/file.lyx and /tmp/file.lyx > in different buffers if, in fact, they are the same file. Yes, that is the case I am aware of. I just tried, and the patch ensures that you cannot open two buffers in that case. Note that there is still a bug in setting path.d->name in FileName::onlyPath(), this needs to be set from path.d->fi, not d->fi. > But maybe the patch works if we store the latter, rather than the > former? Then again, it's hard to check whether we have a canonical path, > so I'll have to think about this. I believe that you do not need to care about canonical paths at all, except when loading .lyx files. The question is whether that assumption is correct. Georg
RE: Re: LyX 1.6.2: possible performance issue inside master document
>> I could be wrong, but I think so, yes. The problem is that an absolute >> filename isn't the same as a canonical filename, i.e., one with >> symlinks etc resolved. We don't want to open /home/me/file.lyx and >> /tmp/file.lyx in different buffers if, in fact, they are the same file. > >Yes, that is the case I am aware of. I just tried, and the patch ensures >that you cannot open two buffers in that case. Note that there is still >a bug in setting path.d->name in FileName::onlyPath(), this needs to be >set from path.d->fi, not d->fi. Oh, and I encounter an assertion when I try to open a file from File->Recent files while already a file A.lyx is open. Then I get an assertion that \path.to\A.lyx\ is not a directory, or is this this bug ? >> But maybe the patch works if we store the latter, rather than the >> former? Then again, it's hard to check whether we have a canonical >> path, so I'll have to think about this. > >I believe that you do not need to care about canonical paths at all, >except when loading .lyx files. The question is whether that assumption >is correct. Wouldn't it be possible to cache the absolute filename of an InsetInclude ? Constructing the absolute filename each time seems like a waste of effort. The only thing that can happen to invalidate the cached value is that the inset points to a link and that link might point to something during the editing of a document. Is that a real possibility that someone would do that ? >Georg Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Georg Baum wrote: I believe that the performance problems could be solved quite easily by basing FileName::operator==() on simple string comparison again and implenting an equivalence() method that is used only when needed. Unfortunately, the case where equivalence() is needed is one of the cases causing the performance problems. h
Re: LyX 1.6.2: possible performance issue inside master document
Georg Baum wrote: I believe that the performance problems could be solved quite easily by basing FileName::operator==() on simple string comparison again and implenting an equivalence() method that is used only when needed. Unfortunately, the case where equivalence() is needed is one of the cases causing the performance problems. h
Re: LyX 1.6.2: possible performance issue inside master document
Hi all, I'm not familar with the coding of LyX in any case - and so don't misinterpret my comment as any kind of paternalism. LyX is a worthy successor of SciWord, open source, and full of excellent ideas and work. I have the same suspicion as Vincent: Each user input inside the master(not only keyboard typing; mouse action, e.g., selecting longer parts of text, does the same!!!) causes some havy checks or updating mechanism related to the child docs. Probably my guess is much too simple: If it is a problem wheather a file is loaded/changed or not (as Vincent mentioned), why not setting a valid-flag as soon as all files are loaded/checked, and or check only files which are not currently loaded or already valid. (Eeach child needs its own flag then). There should be a limited number of user actions, which can made an already checked file invalid. I think a similar mechanism can be applied for each list of the outliner (if not already existent). Hard disk: I have the effect already seen on 2 Laptops, and 1 Standard PC (all WinXP). So I don't think it's related to the HD. Additionally, the HD-Light only flashes once (very short) after the first key is pressed and then not again. But typing remains unuseable slow. This hints, that there is a file system access each time I start typing, but then? Processor load while fast typing inside the master: significant higher, but below 50%; does not increase while fast typing; so I assume slowness is not only related to pure CPU usage Memory usage: far below any limit Hope it help's Zardoz -- Aufgepasst: Sind Ihre Daten beim Online-Banking auch optimal geschützt? Jetzt absichern: https://homebanking.gmx.net/?mc=m...@footer.hb
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: Vincent van Ravesteijn - TNW wrote: Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? It's there that the check is made that the file is loaded in a buffer, which (if I'm remembering right) calls == to see. It seems to me that there are only a few occasions when a file could need to be loaded if it wasn't before. E.g., if an InsetInclude has been added or modified; or if the default master has been changed. So maybe we could have some flag, maybe in the BufferList, that meant: you don't need to check this, nothing has changed. And it could be set and cleared by e.g. InsetInclude constructors. Wouldn't it be better to remove the check completely? Instead of checking if we need to call loadIfNeeded all the time - don't. Instead, check (and loasd) only when an insetinclude is changed/modified. That way, the test don't merely get cheaper, the test goes away which is even better. I don't think there are that many cases to consider. And of course, similar problems could arise elsewhere, and they'd need similar solutions. But it bothers me that we're checking whether every needed file is loaded every time we type a key. Seems excessive indeed. Ideally, typing text should be fine even with a sleep(1) in every filesystem interface. Just a slight stop whenever the emergency save happens. We can't assume that filesystem access will be quick. Helge Hafting
RE: LyX 1.6.2: possible performance issue inside master document
Seems excessive indeed. Ideally, typing text should be fine even with a sleep(1) in every filesystem interface. Just a slight stop whenever the emergency save happens. That's why I now use a test document that I access via WebDav. It is very rewarding to see the speed up then. Helge Hafting Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Helge Hafting wrote: rgheck wrote: Vincent van Ravesteijn - TNW wrote: Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? It's there that the check is made that the file is loaded in a buffer, which (if I'm remembering right) calls == to see. It seems to me that there are only a few occasions when a file could need to be loaded if it wasn't before. E.g., if an InsetInclude has been added or modified; or if the default master has been changed. So maybe we could have some flag, maybe in the BufferList, that meant: you don't need to check this, nothing has changed. And it could be set and cleared by e.g. InsetInclude constructors. Wouldn't it be better to remove the check completely? Instead of checking if we need to call loadIfNeeded all the time - don't. Instead, check (and loasd) only when an insetinclude is changed/modified. That way, the test don't merely get cheaper, the test goes away which is even better. That's roughly my idea, but there are other things that could happen, I think. A child, or master, could be closed for some reason; the master could be saved under a new name; and so forth. So there are two options: (i) Call loadIfNeeded() on all those occasions; (ii) Set a flag on those occasions, which can be checked at the beginning of loadIfNeeded(), and then the check is skipped if it's not set. I prefer (ii) only because I can well imagine other reasons to want to check this flag. (The flag would mean something like: The loaded buffers and/or master-child relationships have been altered.) Richard
RE: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). If you do it like that please be careful that the normalized name is only used for checking equivalence. In the past we invested a lot of work to ensure that file names stored in .lyx files stay always exactly like the user entered them (relative, absolute, case etc). This is IMHO important, think e.g. of documents that are both edited on machines with case sensitive and on others with case insensitive file systems. In other cases it is important that relative names stay relative and absolute ones stay absolute. BTW, I am pretty sure that you do only need to check the equivalence of two filenames in a few cases (e.g. when opening .lyx files). For example, if you have the same file in two graphics insets using different names, you do not really need to know that both files are the same. Of course that may result in two conversions, but given the difficulties with the equivalence check and caching it might even be faster accumulated over the whole editing session. Or if the user changes the included file, and e.g. only changes the case, the document should be marked modified. It would be very annoying if LyX was so clever to recognize that both names are equivalent and not let the user save the document. I believe that the performance problems could be solved quite easily by basing FileName::operator==() on simple string comparison again and implenting an equivalence() method that is used only when needed. This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Why is the class called FileName by the way ? Because the original purpose of this class was to store file names (and not more). Before this class was introduced, file names were stored as plain strings. Sometimes they were absolute, sometimes relative. To make it worse, the base directory of relative names was not always clear. Therefore that class was introduced, which encapsulated a file name and made sure that the base directory of relative names was always known (and it did some minor other stuff, too). During the transition to unicode it was extended to handle filename encoding conversions, too. This really sounds like a class that governs the FileNAME and all related things. And QFileInfo really sounds like things that have to do with the physical file. Exactly. Unfortunately the current FileName class mixes these things. Anyway, I want to have a class which can freely be instantiated and another class with the warning that it can have performance issues because it's related to files on the disk. We might duplicate things, but with our own simple FileName class we are _sure_ that we don't access the disk. By the way, to convert filenames and stuff we still can use QFileInfo to be cross-platform. We just use it once, and when the filename is in our own class, we are sure that it has the correct format. That was the idea. What you describe is exactly the orginal idea of the FileName class and can be seen e.g. in the 1.5.x sources. Georg
Re: LyX 1.6.2: possible performance issue inside master document
Hi all, I'm not familar with the coding of LyX in any case - and so don't misinterpret my comment as any kind of paternalism. LyX is a worthy successor of SciWord, open source, and full of excellent ideas and work. I have the same suspicion as Vincent: Each user input inside the master(not only keyboard typing; mouse action, e.g., selecting longer parts of text, does the same!!!) causes some havy checks or updating mechanism related to the child docs. Probably my guess is much too simple: If it is a problem wheather a file is loaded/changed or not (as Vincent mentioned), why not setting a valid-flag as soon as all files are loaded/checked, and or check only files which are not currently loaded or already valid. (Eeach child needs its own flag then). There should be a limited number of user actions, which can made an already checked file invalid. I think a similar mechanism can be applied for each "list" of the outliner (if not already existent). Hard disk: I have the effect already seen on 2 Laptops, and 1 Standard PC (all WinXP). So I don't think it's related to the HD. Additionally, the HD-Light only flashes once (very short) after the first key is pressed and then not again. But typing remains unuseable slow. This hints, that there is a file system access each time I start typing, but then? Processor load while fast typing inside the master: significant higher, but below 50%; does not increase while fast typing; so I assume slowness is not only related to pure CPU usage Memory usage: far below any limit Hope it help's Zardoz -- Aufgepasst: Sind Ihre Daten beim Online-Banking auch optimal geschützt? Jetzt absichern: https://homebanking.gmx.net/?mc=m...@footer.hb
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: Vincent van Ravesteijn - TNW wrote: Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? It's there that the check is made that the file is loaded in a buffer, which (if I'm remembering right) calls == to see. It seems to me that there are only a few occasions when a file could need to be loaded if it wasn't before. E.g., if an InsetInclude has been added or modified; or if the default master has been changed. So maybe we could have some flag, maybe in the BufferList, that meant: you don't need to check this, nothing has changed. And it could be set and cleared by e.g. InsetInclude constructors. Wouldn't it be better to remove the check completely? Instead of checking if we need to call "loadIfNeeded" all the time - don't. Instead, check (and loasd) only when an insetinclude is changed/modified. That way, the test don't merely get cheaper, the test goes away which is even better. I don't think there are that many cases to consider. And of course, similar problems could arise elsewhere, and they'd need similar solutions. But it bothers me that we're checking whether every needed file is loaded every time we type a key. Seems excessive indeed. Ideally, typing text should be fine even with a "sleep(1)" in every filesystem interface. Just a slight stop whenever the emergency save happens. We can't assume that filesystem access will be quick. Helge Hafting
RE: LyX 1.6.2: possible performance issue inside master document
>Seems excessive indeed. Ideally, typing text should be fine >even with a "sleep(1)" in every filesystem interface. Just a >slight stop whenever the emergency save happens. That's why I now use a test document that I access via WebDav. It is very rewarding to see the speed up then. >Helge Hafting Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Helge Hafting wrote: rgheck wrote: Vincent van Ravesteijn - TNW wrote: Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? It's there that the check is made that the file is loaded in a buffer, which (if I'm remembering right) calls == to see. It seems to me that there are only a few occasions when a file could need to be loaded if it wasn't before. E.g., if an InsetInclude has been added or modified; or if the default master has been changed. So maybe we could have some flag, maybe in the BufferList, that meant: you don't need to check this, nothing has changed. And it could be set and cleared by e.g. InsetInclude constructors. Wouldn't it be better to remove the check completely? Instead of checking if we need to call "loadIfNeeded" all the time - don't. Instead, check (and loasd) only when an insetinclude is changed/modified. That way, the test don't merely get cheaper, the test goes away which is even better. That's roughly my idea, but there are other things that could happen, I think. A child, or master, could be closed for some reason; the master could be saved under a new name; and so forth. So there are two options: (i) Call loadIfNeeded() on all those occasions; (ii) Set a flag on those occasions, which can be checked at the beginning of loadIfNeeded(), and then the check is skipped if it's not set. I prefer (ii) only because I can well imagine other reasons to want to check this flag. (The flag would mean something like: The loaded buffers and/or master-child relationships have been altered.) Richard
RE: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: >>> Do we need a new class that we use internally. In this class we store > >>> a filename, which is case-sensitive on case-sensitive filesystem, > that >>> is guaranteed to be a long filename and not a short one (on Windows), > >>> that is not a symlink, that is relative or absolute (pick one or > both). If you do it like that please be careful that the normalized name is only used for checking equivalence. In the past we invested a lot of work to ensure that file names stored in .lyx files stay always exactly like the user entered them (relative, absolute, case etc). This is IMHO important, think e.g. of documents that are both edited on machines with case sensitive and on others with case insensitive file systems. In other cases it is important that relative names stay relative and absolute ones stay absolute. BTW, I am pretty sure that you do only need to check the equivalence of two filenames in a few cases (e.g. when opening .lyx files). For example, if you have the same file in two graphics insets using different names, you do not really need to know that both files are the same. Of course that may result in two conversions, but given the difficulties with the equivalence check and caching it might even be faster accumulated over the whole editing session. Or if the user changes the included file, and e.g. only changes the case, the document should be marked modified. It would be very annoying if LyX was so "clever" to recognize that both names are equivalent and not let the user save the document. I believe that the performance problems could be solved quite easily by basing FileName::operator==() on simple string comparison again and implenting an equivalence() method that is used only when needed. >>This could work, though it looks as if we're duplicating what QFileInfo >>does, and we might run into similar problems about caching to what we >>had before. I don't know---but I think this makes some sense out of why >>they want to return false if the files don't exist. The problem, as I > see >>it, is that QFileInfo objects are getting created so often, which > requires >>filesystem access, even independently of the caching stuff. And I'm not >>sure how that can be avoided. > > Why is the class called FileName by the way ? Because the original purpose of this class was to store file names (and not more). Before this class was introduced, file names were stored as plain strings. Sometimes they were absolute, sometimes relative. To make it worse, the base directory of relative names was not always clear. Therefore that class was introduced, which encapsulated a file name and made sure that the base directory of relative names was always known (and it did some minor other stuff, too). During the transition to unicode it was extended to handle filename encoding conversions, too. > This really sounds like a > class that governs the FileNAME and all related things. And QFileInfo > really sounds like things that have to do with the physical file. Exactly. Unfortunately the current FileName class mixes these things. > Anyway, I want to have a class which can freely be instantiated and > another class with the warning that it can have performance issues > because it's related to files on the disk. We might duplicate things, > but with our own simple FileName class we are _sure_ that we don't > access the disk. By the way, to convert filenames and stuff we still can > use QFileInfo to be cross-platform. We just use it once, and when the > filename is in our own class, we are sure that it has the correct > format. > > That was the idea. What you describe is exactly the orginal idea of the FileName class and can be seen e.g. in the 1.5.x sources. Georg
Re: LyX 1.6.2: possible performance issue inside master document
On 2009-03-18, Richard Heck wrote: pseudo2...@gmx.at wrote: Child document opened alone or together with its parent? For me it looks like heavy internal updating inside the master is done for each included child-doc (it does not matter whether the child is opend). I wish I had some idea why this is happening only on certain platforms. Or is it only on certain platforms? I've not seen reports of this behavior under Linux, but maybe there are people with similar problems. Maybe it is because 1.6.2 is not yet available for Debian? Or because many people are editing the master less frequently than its children? I see a slower response in a master than in a child also under Linux. (Debian/testing, LyX 1.6.2-svn). It is still usable but annoying. Maybe it gets worse with auto-completion (which I turned off for performance resons). Günter
Re: LyX 1.6.2: possible performance issue inside master document
If you are able to send me a test document (maybe privately) I'll have a look. Even better: put it in a new bugzilla entry. I've created some testdocument that reproduces the behaviour (where can files be uploaded?). (WinXP) - 1.6.1 (AltInstaller or Classic same behaviour): slow down inside master not too dramatic; worsens as soon as outline is turned on - 1.6.2 slow down realy bad; normal typing or marking with mouse inside the master impossible; surprisingly processor load only 50% an does not increase while characters appear in slow motion I have not seen any network traffic load I hope it will help, Zardoz -- Aufgepasst: Sind Ihre Daten beim Online-Banking auch optimal geschützt? Jetzt absichern: https://homebanking.gmx.net/?mc=m...@footer.hb
Re: LyX 1.6.2: possible performance issue inside master document
pseudo2...@gmx.at wrote: I've created some testdocument that reproduces the behaviour (where can files be uploaded?). Just attach it. rh
Re: LyX 1.6.2: possible performance issue inside master document
pseudo2...@gmx.at wrote: Well, my attached zip-file has been rejected. I try the lyx-files. An no, the latex view source window is not open. OK, thanks. As a data point, I don't see any slowdown in Linux, using latest branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when (unsurprisingly) I do. Same result if I put the files on a network drive. Is it possible that LaTeX is being generated, even though the window isn't shown? Richard
Re: LyX 1.6.2: possible performance issue inside master document
Well list didn't accept the large files. Now with small ones. Slowness is the same. Thanks, Zardoz Original-Nachricht Datum: Thu, 19 Mar 2009 12:31:59 -0400 Von: rgheck rgh...@bobjweil.com An: pseudo2...@gmx.at CC: lyx-devel@lists.lyx.org Betreff: Re: LyX 1.6.2: possible performance issue inside master document pseudo2...@gmx.at wrote: Well, my attached zip-file has been rejected. I try the lyx-files. An no, the latex view source window is not open. OK, thanks. As a data point, I don't see any slowdown in Linux, using latest branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when (unsurprisingly) I do. Same result if I put the files on a network drive. Is it possible that LaTeX is being generated, even though the window isn't shown? Richard -- Pt! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01 testChild1.lyx Description: Binary data testChild2.lyx Description: Binary data testChild3.lyx Description: Binary data testChild4.lyx Description: Binary data testChild5.lyx Description: Binary data testChild6.lyx Description: Binary data testChild7.lyx Description: Binary data testChild51.lyx Description: Binary data testChild52.lyx Description: Binary data testMaster.lyx Description: Binary data
RE: LyX 1.6.2: possible performance issue inside master document
As a data point, I don't see any slowdown in Linux, using latest branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when (unsurprisingly) I do. Same result if I put the files on a network drive. Is it possible that LaTeX is being generated, even though the window isn't shown? I don't think LaTeX code generation is the problem. All reports are speaking about a master document with childs. Usually the LaTeX code is then nothing else than a few includes and maybe a title, toc, references and so. Don't you believe my reasoning about the thousands file accesses per second ? Anyway, a possible reason why it is worse at windows are the following lines of code: FileName.cpp: 937 FileName const lhs(os::internal_path(l.absFilename())); 938 FileName const rhs(os::internal_path(r.absFilename())); In os_win32.cpp::get_long_path the function GetLongPathName is called. This function checks whether the file exists. (And then we do it again in the constructor of FileName). So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. And now some speculation: I'm using UNC-paths. That has caused problems in the past I believe. Maybe the fi.exists() function somehow returns false, or we use a wrong representation of the path somewhere. If then, caching is turned on in FileName(), then the problem gets worse again. By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like Abdel's hammer. Richard Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: As a data point, I don't see any slowdown in Linux, using latest branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when (unsurprisingly) I do. Same result if I put the files on a network drive. Is it possible that LaTeX is being generated, even though the window isn't shown? I don't think LaTeX code generation is the problem. All reports are speaking about a master document with childs. Usually the LaTeX code is then nothing else than a few includes and maybe a title, toc, references and so. Don't you believe my reasoning about the thousands file accesses per second ? No, it's not that I don't believe it, but I'm still fishing for possible reasons. There may be more than one. And I'm also still puzzled why it's not happening on Linux, in particular, here. I've got all my files on a network drive as well, and still see nothing. That led me to wonder whether there's something going on with Qt. But it's just a guess. And Anyway, a possible reason why it is worse at windows are the following lines of code: FileName.cpp: 937 FileName const lhs(os::internal_path(l.absFilename())); 938 FileName const rhs(os::internal_path(r.absFilename())); In os_win32.cpp::get_long_path the function GetLongPathName is called. This function checks whether the file exists. (And then we do it again in the constructor of FileName). So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. That would make it worse. But enough worse that I see nothing here? Is network file system performance much slower under Windows? Can some of these checks be turned off, just as a test? And now some speculation: I'm using UNC-paths. That has caused problems in the past I believe. Maybe the fi.exists() function somehow returns false, or we use a wrong representation of the path somewhere. If then, caching is turned on in FileName(), then the problem gets worse again. By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like Abdel's hammer. Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. rh
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like Abdel's hammer. Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. http://marc.info/?l=lyx-develm=121734075326705w=2 rh
RE: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like Abdel's hammer. Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. http://marc.info/?l=lyx-develm=121734075326705w=2 rh The thread starts with a problem that would be the consequence of r25960. I don't see why refreshing is a solution for a problem that could have been introduces in r25960. The thread neither gives an answer why and whether the changes are correct. More question marks thus.. Vincent
RE: LyX 1.6.2: possible performance issue inside master document
So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. That would make it worse. But enough worse that I see nothing here? Is network file system performance much slower under Windows? Can some of these checks be turned off, just as a test? If I turn off both the os::internal_path calls and the refresh calls, LyX is running smoothly. At least for me it is. I do still wonder why the other people get the delay. They either have just a very slow harddisk or there must be something else. Another difference is the following piece of code: 117 #if defined(_WIN32) || (QT_VERSION = 0x99) 118 fi.refresh(); 119 #else 120 fi = QFileInfo(fi.absoluteFilePath()); 121 #endif Maybe, fi = QFileInfo(fi.absoluteFilePath()) is faster than fi.refresh(). I could try to use that for Windows too and see whether it makes a difference, but I can't do that now. rh Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: rgheck wrote: By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like Abdel's hammer. Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. http://marc.info/?l=lyx-develm=121734075326705w=2 rh The thread starts with a problem that would be the consequence of r25960. I don't see why refreshing is a solution for a problem that could have been introduces in r25960. The thread neither gives an answer why and whether the changes are correct. The previous == and != checks had just been against the file NAMES, which meant you could get file1 != file2, if the names were symlinks pointing to the same file, and names with different capatalizations would be seen as different on Windows, VFAT, etc. So Abdel changed it to call QFileInfo::operator==, which dealt with both problems. But that led to Bennett's problem: The comparison sometimes rested on old data; hence the need for the refresh. rh
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. That would make it worse. But enough worse that I see nothing here? Is network file system performance much slower under Windows? Can some of these checks be turned off, just as a test? If I turn off both the os::internal_path calls and the refresh calls, LyX is running smoothly. Well, that's at least some progress. At least for me it is. I do still wonder why the other people get the delay. They either have just a very slow harddisk or there must be something else. I find this very puzzling. We've seen some weird things involving slow drawing before, but it's hard to see why that would happen with children, etc. Of course, it'd be a lot easier to figure this out if I could reproduce it. Maybe I'll try it on my wife's machine later. Then I'll at least be able to see it happen for myself. Another difference is the following piece of code: 117 #if defined(_WIN32) || (QT_VERSION = 0x99) 118 fi.refresh(); 119 #else 120 fi = QFileInfo(fi.absoluteFilePath()); 121 #endif Maybe, fi = QFileInfo(fi.absoluteFilePath()) is faster than fi.refresh(). Hard to see why it would be. The construction of a QFileInfo object presumably accesses the file system in the same way? Though maybe it's lazy and just waits to access when it needs to. rh
RE: LyX 1.6.2: possible performance issue inside master document
-Original Message- From: rgheck [mailto:rgh...@bobjweil.com] Sent: donderdag 19 maart 2009 20:46 To: Vincent van Ravesteijn - TNW Cc: lyx-devel@lists.lyx.org Subject: Re: LyX 1.6.2: possible performance issue inside master document Vincent van Ravesteijn - TNW wrote: rgheck wrote: By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like Abdel's hammer. Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. http://marc.info/?l=lyx-develm=121734075326705w=2 rh The thread starts with a problem that would be the consequence of r25960. I don't see why refreshing is a solution for a problem that could have been introduces in r25960. The thread neither gives an answer why and whether the changes are correct. The previous == and != checks had just been against the file NAMES, which meant you could get file1 != file2, if the names were symlinks pointing to the same file, and names with different capatalizations would be seen as different on Windows, VFAT, etc. So Abdel changed it to call QFileInfo::operator==, which dealt with both problems. But that led to Bennett's problem: The comparison sometimes rested on old data; hence the need for the refresh. The old way of checking the file NAMES would also have been based on 'old' data or not ? That's what I don't understand, was it like this: A file should be copied when the target fileNAME was different from the source fileNAME ? When changing to the ==() operator, we checked whether the two files are both existent and have exactly the same name and _size_ (that is used in QFileInfo::operator==()). Thus if one of the two objects has a cached value that the file did not exist and the other (a new FileName object) says the file does exist. ==() returns false for the two files. Then copying a file onto itself will fail. But this shows an example in which it is of _no_ interest whether the files are exactly the same. It is only of interest to know whether the path is the same, so QFileInfo::==() should not be used. Does this make sense ? rh Vincent
Re: LyX 1.6.2: possible performance issue inside master document
The previous == and != checks had just been against the file NAMES, which meant you could get file1 != file2, if the names were symlinks pointing to the same file, and names with different capatalizations would be seen as different on Windows, VFAT, etc. So Abdel changed it to call QFileInfo::operator==, which dealt with both problems. But that led to Bennett's problem: The comparison sometimes rested on old data; hence the need for the refresh. The old way of checking the file NAMES would also have been based on 'old' data or not ? No. Because it just checked the names, whereas the new one used QFileInfo::operator==. And QFileInfo would cache the data, so it could be out of date. See below That's what I don't understand, was it like this: A file should be copied when the target fileNAME was different from the source fileNAME? When changing to the ==() operator, we checked whether the two files are both existent and have exactly the same name and _size_ (that is used in QFileInfo::operator==()). Not quite, because QFileInfo::operator== is supposed to return true if they're the SAME FILE, independent of whether they have the SAME NAME. The size check seems like something they do to avoid checking the more complicated stuff, viz., canonicalFilePath(). I guess it must be cheaper. But what they're checking for is whether it's the same file. Thus if one of the two objects has a cached value that the file did not exist and the other (a new FileName object) says the file does exist. ==() returns false for the two files. Then copying a file onto itself will fail. But this shows an example in which it is of _no_ interest whether the files are exactly the same. It is only of interest to know whether the path is the same, so QFileInfo::==() should not be used. Does this make sense ? Maybe it's already clear, but the point now is that it isn't enough to check if the *strings* giving the paths are the same. That's what we used to do, and it was behind the bugs Abdel was trying to fix. E.g., suppose you have a buffer open with file.lyx. Now you ask LyX to open FILE.lyx. Previously, LyX would have done this, since it didn't realize the files are the same. And now you're editing the same file twice, and LyX is happily overwriting your changes. Worse, imagine you have newfile1.lyx, and now you create a new file which you ask LyX to save as NewFile1.lyx. You can get similar problems if file.lyx is a symlink pointing at otherfile.lyx. The solution to this, again, was to use QFileInfo, which uses canonicalFilePath() to avoid these kinds of problems. The problem Bennett saw was due to failure of one of the checks (in trunk) in Converter.cpp, line 364 or 500, if I remember right, due to caching. I'd have to look more closely to be sure, but I think those checks had better be for file equality, not just string equality, lest you overwrite something you don't mean to be overwriting. Richard
RE: LyX 1.6.2: possible performance issue inside master document
Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). Ideally, we can now in most cases just compare the strings. Only when we have to deal with user-defined paths we need to convert it into our standard format, or we ask QFileInfo whether a file already exists etc. (PS. The new class we should then call FileName and the current FileName class we should call File or FileInfo (maybe later)). Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? It's there that the check is made that the file is loaded in a buffer, which (if I'm remembering right) calls == to see. It seems to me that there are only a few occasions when a file could need to be loaded if it wasn't before. E.g., if an InsetInclude has been added or modified; or if the default master has been changed. So maybe we could have some flag, maybe in the BufferList, that meant: you don't need to check this, nothing has changed. And it could be set and cleared by e.g. InsetInclude constructors. I don't think there are that many cases to consider. And of course, similar problems could arise elsewhere, and they'd need similar solutions. But it bothers me that we're checking whether every needed file is loaded every time we type a key. And, while we're at it, maybe a similar solution could work for updateMacros(). I don't know well enough what that does to be sure, but it seems reasonable to suppose that there are only certain events that could invalidate the information that routine gathers. So we have a flag, this time in the Buffer, that means: you don't have to update the macros. And it gets cleared by things that make you want to update them. Come to think of it, I implemented a not totally dissimilar thing with Buffer::invalidateBibinfoCache(). This doesn't prevent filesystem checks---we don't control whether bib files might have changed---but it allows insets to force invalidation of the cache and force a reload whenever the user might have changed the list of bibfiles, etc. Richard
RE: LyX 1.6.2: possible performance issue inside master document
Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Why is the class called FileName by the way ? This really sounds like a class that governs the FileNAME and all related things. And QFileInfo really sounds like things that have to do with the physical file. Anyway, I want to have a class which can freely be instantiated and another class with the warning that it can have performance issues because it's related to files on the disk. We might duplicate things, but with our own simple FileName class we are _sure_ that we don't access the disk. By the way, to convert filenames and stuff we still can use QFileInfo to be cross-platform. We just use it once, and when the filename is in our own class, we are sure that it has the correct format. That was the idea. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? True, that should be fixed too. But it bothers me that we're checking whether every needed file is loaded every time we type a key. Me too. And, while we're at it, maybe a similar solution could work for updateMacros(). Yes, you might remember I was complaining about the infinite number of calls to updateMacros a while ago. If you think about it, why would we want to call the updateMacros function of a child document. Like that could have changed by pressing a character in the master document ?!? I hope other devs also have an opinion on this, but I guess we should fix it all of the above. Richard Vincent
Re: LyX 1.6.2: possible performance issue inside master document
On 2009-03-18, Richard Heck wrote: > pseudo2...@gmx.at wrote: >>> Child document opened alone or together with its parent? >> For me it looks like heavy internal updating inside the master is done >> for each included child-doc (it does not matter whether the child is >> opend). > I wish I had some idea why this is happening only on certain platforms. > Or is it only on certain platforms? I've not seen reports of this > behavior under Linux, but maybe there are people with similar problems. Maybe it is because 1.6.2 is not yet available for Debian? Or because many people are editing the master less frequently than its children? I see a slower response in a master than in a child also under Linux. (Debian/testing, LyX 1.6.2-svn). It is still usable but annoying. Maybe it gets worse with auto-completion (which I turned off for performance resons). Günter
Re: LyX 1.6.2: possible performance issue inside master document
> If you are able to send me a test document (maybe privately) I'll have a > look. Even better: put it in a new bugzilla entry. > I've created some testdocument that reproduces the behaviour (where can files be uploaded?). (WinXP) -> 1.6.1 (AltInstaller or Classic same behaviour): slow down inside master not too dramatic; worsens as soon as "outline" is turned on -> 1.6.2 slow down realy bad; normal typing or marking with mouse inside the master impossible; surprisingly processor load "only" 50% an does not increase while characters appear in "slow motion" I have not seen any network traffic load I hope it will help, Zardoz -- Aufgepasst: Sind Ihre Daten beim Online-Banking auch optimal geschützt? Jetzt absichern: https://homebanking.gmx.net/?mc=m...@footer.hb
Re: LyX 1.6.2: possible performance issue inside master document
pseudo2...@gmx.at wrote: I've created some testdocument that reproduces the behaviour (where can files be uploaded?). Just attach it. rh
Re: LyX 1.6.2: possible performance issue inside master document
pseudo2...@gmx.at wrote: Well, my attached zip-file has been rejected. I try the lyx-files. An no, the latex "view source" window is not open. OK, thanks. As a data point, I don't see any slowdown in Linux, using latest branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when (unsurprisingly) I do. Same result if I put the files on a network drive. Is it possible that LaTeX is being generated, even though the window isn't shown? Richard
Re: LyX 1.6.2: possible performance issue inside master document
Well list didn't accept the large files. Now with small ones. Slowness is the same. Thanks, Zardoz Original-Nachricht > Datum: Thu, 19 Mar 2009 12:31:59 -0400 > Von: rgheck <rgh...@bobjweil.com> > An: pseudo2...@gmx.at > CC: lyx-devel@lists.lyx.org > Betreff: Re: LyX 1.6.2: possible performance issue inside master document > pseudo2...@gmx.at wrote: > > Well, my attached zip-file has been rejected. I try the lyx-files. > > > > An no, the latex "view source" window is not open. > > > > > OK, thanks. > > As a data point, I don't see any slowdown in Linux, using latest branch > and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when > (unsurprisingly) I do. Same result if I put the files on a network drive. > > Is it possible that LaTeX is being generated, even though the window > isn't shown? > > Richard -- Pt! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01 testChild1.lyx Description: Binary data testChild2.lyx Description: Binary data testChild3.lyx Description: Binary data testChild4.lyx Description: Binary data testChild5.lyx Description: Binary data testChild6.lyx Description: Binary data testChild7.lyx Description: Binary data testChild51.lyx Description: Binary data testChild52.lyx Description: Binary data testMaster.lyx Description: Binary data
RE: LyX 1.6.2: possible performance issue inside master document
>As a data point, I don't see any slowdown in Linux, using latest >branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source >window, when (unsurprisingly) I do. Same result if I put the files >on a network drive. > >Is it possible that LaTeX is being generated, even though the window >isn't shown? > I don't think LaTeX code generation is the problem. All reports are speaking about a master document with childs. Usually the LaTeX code is then nothing else than a few includes and maybe a title, toc, references and so. Don't you believe my reasoning about the thousands file accesses per second ? Anyway, a possible reason why it is worse at windows are the following lines of code: FileName.cpp: 937 FileName const lhs(os::internal_path(l.absFilename())); 938 FileName const rhs(os::internal_path(r.absFilename())); In os_win32.cpp::get_long_path the function GetLongPathName is called. This function checks whether the file exists. (And then we do it again in the constructor of FileName). So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. And now some speculation: I'm using UNC-paths. That has caused problems in the past I believe. Maybe the fi.exists() function somehow returns false, or we use a wrong representation of the path somewhere. If then, caching is turned on in FileName(), then the problem gets worse again. By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like "Abdel's hammer". >Richard Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: As a data point, I don't see any slowdown in Linux, using latest branch and Qt 4.4.3, under Fedora 8. UNLESS I open the View Source window, when (unsurprisingly) I do. Same result if I put the files on a network drive. Is it possible that LaTeX is being generated, even though the window isn't shown? I don't think LaTeX code generation is the problem. All reports are speaking about a master document with childs. Usually the LaTeX code is then nothing else than a few includes and maybe a title, toc, references and so. Don't you believe my reasoning about the thousands file accesses per second ? No, it's not that I don't believe it, but I'm still fishing for possible reasons. There may be more than one. And I'm also still puzzled why it's not happening on Linux, in particular, here. I've got all my files on a network drive as well, and still see nothing. That led me to wonder whether there's something going on with Qt. But it's just a guess. And Anyway, a possible reason why it is worse at windows are the following lines of code: FileName.cpp: 937 FileName const lhs(os::internal_path(l.absFilename())); 938 FileName const rhs(os::internal_path(r.absFilename())); In os_win32.cpp::get_long_path the function GetLongPathName is called. This function checks whether the file exists. (And then we do it again in the constructor of FileName). So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. That would make it worse. But enough worse that I see nothing here? Is network file system performance much slower under Windows? Can some of these checks be turned off, just as a test? And now some speculation: I'm using UNC-paths. That has caused problems in the past I believe. Maybe the fi.exists() function somehow returns false, or we use a wrong representation of the path somewhere. If then, caching is turned on in FileName(), then the problem gets worse again. By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like "Abdel's hammer". Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. rh
Re: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like "Abdel's hammer". Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. http://marc.info/?l=lyx-devel=121734075326705=2 rh
RE: LyX 1.6.2: possible performance issue inside master document
rgheck wrote: >>> By the way, can you remember what problem Bennet had which made you >>> adding the lhs.refresh() and rhs.refresh() lines to >>> FileName::operator==(). This solution looks a bit like "Abdel's hammer". >>> >>> >> Not right now, no, though it's probably on the list somewhere. But I >> think the problem was that, when caching is enabled, the file could >> have moved, been deleted, etc, and you'll miss it without a refresh. >> >http://marc.info/?l=lyx-devel=121734075326705=2 > >rh > The thread starts with a problem that would be the consequence of r25960. I don't see why refreshing is a solution for a problem that could have been introduces in r25960. The thread neither gives an answer why and whether the changes are correct. More question marks thus.. Vincent
RE: LyX 1.6.2: possible performance issue inside master document
>> So, if there are four filesystem-checks in Linux, there are at least >> six on Windows and maybe the compiler does something smart when >> os::internal_path does nothing. >> >> >That would make it worse. But enough worse that I see nothing >here? Is network file system performance much >slower under >Windows? Can some of these checks be turned off, just as a test? If I turn off both the os::internal_path calls and the refresh calls, LyX is running smoothly. At least for me it is. I do still wonder why the other people get the delay. They either have just a very slow harddisk or there must be something else. Another difference is the following piece of code: 117 #if defined(_WIN32) || (QT_VERSION >= 0x99) 118 fi.refresh(); 119 #else 120 fi = QFileInfo(fi.absoluteFilePath()); 121 #endif Maybe, fi = QFileInfo(fi.absoluteFilePath()) is faster than fi.refresh(). I could try to use that for Windows too and see whether it makes a difference, but I can't do that now. > rh Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: rgheck wrote: By the way, can you remember what problem Bennet had which made you adding the lhs.refresh() and rhs.refresh() lines to FileName::operator==(). This solution looks a bit like "Abdel's hammer". Not right now, no, though it's probably on the list somewhere. But I think the problem was that, when caching is enabled, the file could have moved, been deleted, etc, and you'll miss it without a refresh. http://marc.info/?l=lyx-devel=121734075326705=2 rh The thread starts with a problem that would be the consequence of r25960. I don't see why refreshing is a solution for a problem that could have been introduces in r25960. The thread neither gives an answer why and whether the changes are correct. The previous == and != checks had just been against the file NAMES, which meant you could get file1 != file2, if the names were symlinks pointing to the same file, and names with different capatalizations would be seen as different on Windows, VFAT, etc. So Abdel changed it to call QFileInfo::operator==, which dealt with both problems. But that led to Bennett's problem: The comparison sometimes rested on old data; hence the need for the refresh. rh
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: So, if there are four filesystem-checks in Linux, there are at least six on Windows and maybe the compiler does something smart when os::internal_path does nothing. That would make it worse. But enough worse that I see nothing here? Is network file system performance much >slower under Windows? Can some of these checks be turned off, just as a test? If I turn off both the os::internal_path calls and the refresh calls, LyX is running smoothly. Well, that's at least some progress. At least for me it is. I do still wonder why the other people get the delay. They either have just a very slow harddisk or there must be something else. I find this very puzzling. We've seen some weird things involving slow drawing before, but it's hard to see why that would happen with children, etc. Of course, it'd be a lot easier to figure this out if I could reproduce it. Maybe I'll try it on my wife's machine later. Then I'll at least be able to see it happen for myself. Another difference is the following piece of code: 117 #if defined(_WIN32) || (QT_VERSION >= 0x99) 118 fi.refresh(); 119 #else 120 fi = QFileInfo(fi.absoluteFilePath()); 121 #endif Maybe, fi = QFileInfo(fi.absoluteFilePath()) is faster than fi.refresh(). Hard to see why it would be. The construction of a QFileInfo object presumably accesses the file system in the same way? Though maybe it's "lazy" and just waits to access when it needs to. rh
RE: LyX 1.6.2: possible performance issue inside master document
-Original Message- From: rgheck [mailto:rgh...@bobjweil.com] Sent: donderdag 19 maart 2009 20:46 To: Vincent van Ravesteijn - TNW Cc: lyx-devel@lists.lyx.org Subject: Re: LyX 1.6.2: possible performance issue inside master document Vincent van Ravesteijn - TNW wrote: > > rgheck wrote: > >>>> By the way, can you remember what problem Bennet had which made you >>>> adding the lhs.refresh() and rhs.refresh() lines to >>>> FileName::operator==(). This solution looks a bit like "Abdel's >>>> > hammer". > >>>> >>>> >>> Not right now, no, though it's probably on the list somewhere. But I >>> think the problem was that, when caching is enabled, the file could >>> have moved, been deleted, etc, and you'll miss it without a refresh. >>> >>> >> http://marc.info/?l=lyx-devel=121734075326705=2 >> >> rh >> >> > > The thread starts with a problem that would be the consequence of > r25960. > > I don't see why refreshing is a solution for a problem that could have > been introduces in r25960. The thread neither gives an answer why and > whether the changes are correct. > > >The previous == and != checks had just been against the file NAMES, which meant you could get file1 != file2, >if the names were symlinks pointing to the same file, and names with different capatalizations would be seen >as different on Windows, VFAT, etc. So Abdel changed it to call QFileInfo::operator==, which dealt with both >problems. But that led to Bennett's problem: The comparison sometimes rested on old data; hence the need for >the refresh. > The old way of checking the file NAMES would also have been based on 'old' data or not ? That's what I don't understand, was it like this: A file should be copied when the target fileNAME was different from the source fileNAME ? When changing to the ==() operator, we checked whether the two files are both existent and have exactly the same name and _size_ (that is used in QFileInfo::operator==()). Thus if one of the two objects has a cached value that the file did not exist and the other (a new FileName object) says the file does exist. ==() returns false for the two files. Then copying a file onto itself will fail. But this shows an example in which it is of _no_ interest whether the files are exactly the same. It is only of interest to know whether the path is the same, so QFileInfo::==() should not be used. Does this make sense ? >rh Vincent
Re: LyX 1.6.2: possible performance issue inside master document
The previous == and != checks had just been against the file NAMES, which meant you could get file1 != file2, >if the names were symlinks pointing to the same file, and names with different capatalizations would be seen >as different on Windows, VFAT, etc. So Abdel changed it to call QFileInfo::operator==, which dealt with both >problems. But that led to Bennett's problem: The comparison sometimes rested on old data; hence the need for >the refresh. The old way of checking the file NAMES would also have been based on 'old' data or not ? No. Because it just checked the names, whereas the new one used QFileInfo::operator==. And QFileInfo would cache the data, so it could be out of date. See below That's what I don't understand, was it like this: A file should be copied when the target fileNAME was different from the source fileNAME? When changing to the ==() operator, we checked whether the two files are both existent and have exactly the same name and _size_ (that is used in QFileInfo::operator==()). Not quite, because QFileInfo::operator== is supposed to return true if they're the SAME FILE, independent of whether they have the SAME NAME. The size check seems like something they do to avoid checking the more complicated stuff, viz., canonicalFilePath(). I guess it must be cheaper. But what they're checking for is whether it's the same file. Thus if one of the two objects has a cached value that the file did not exist and the other (a new FileName object) says the file does exist. ==() returns false for the two files. Then copying a file onto itself will fail. But this shows an example in which it is of _no_ interest whether the files are exactly the same. It is only of interest to know whether the path is the same, so QFileInfo::==() should not be used. Does this make sense ? Maybe it's already clear, but the point now is that it isn't enough to check if the *strings* giving the paths are the same. That's what we used to do, and it was behind the bugs Abdel was trying to fix. E.g., suppose you have a buffer open with file.lyx. Now you ask LyX to open FILE.lyx. Previously, LyX would have done this, since it didn't realize the files are the same. And now you're editing the same file twice, and LyX is happily overwriting your changes. Worse, imagine you have newfile1.lyx, and now you create a new file which you ask LyX to save as NewFile1.lyx. You can get similar problems if file.lyx is a symlink pointing at otherfile.lyx. The solution to this, again, was to use QFileInfo, which uses canonicalFilePath() to avoid these kinds of problems. The problem Bennett saw was due to failure of one of the checks (in trunk) in Converter.cpp, line 364 or 500, if I remember right, due to caching. I'd have to look more closely to be sure, but I think those checks had better be for file equality, not just string equality, lest you overwrite something you don't mean to be overwriting. Richard
RE: LyX 1.6.2: possible performance issue inside master document
> Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). Ideally, we can now in most cases just compare the strings. Only when we have to deal with user-defined paths we need to convert it into our standard format, or we ask QFileInfo whether a file already exists etc. (PS. The new class we should then call FileName and the current FileName class we should call File or FileInfo (maybe later)). Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Vincent van Ravesteijn - TNW wrote: Maybe it's already clear, Yes, I understand now. How do we solve it then ? Do we need a new class that we use internally. In this class we store a filename, which is case-sensitive on case-sensitive filesystem, that is guaranteed to be a long filename and not a short one (on Windows), that is not a symlink, that is relative or absolute (pick one or both). This could work, though it looks as if we're duplicating what QFileInfo does, and we might run into similar problems about caching to what we had before. I don't know---but I think this makes some sense out of why they want to return false if the files don't exist. The problem, as I see it, is that QFileInfo objects are getting created so often, which requires filesystem access, even independently of the caching stuff. And I'm not sure how that can be avoided. Anyway, here's another idea. Why are we calling loadIfNeeded() every time? It's there that the check is made that the file is loaded in a buffer, which (if I'm remembering right) calls == to see. It seems to me that there are only a few occasions when a file could need to be loaded if it wasn't before. E.g., if an InsetInclude has been added or modified; or if the default master has been changed. So maybe we could have some flag, maybe in the BufferList, that meant: you don't need to check this, nothing has changed. And it could be set and cleared by e.g. InsetInclude constructors. I don't think there are that many cases to consider. And of course, similar problems could arise elsewhere, and they'd need similar solutions. But it bothers me that we're checking whether every needed file is loaded every time we type a key. And, while we're at it, maybe a similar solution could work for updateMacros(). I don't know well enough what that does to be sure, but it seems reasonable to suppose that there are only certain events that could invalidate the information that routine gathers. So we have a flag, this time in the Buffer, that means: you don't have to update the macros. And it gets cleared by things that make you want to update them. Come to think of it, I implemented a not totally dissimilar thing with Buffer::invalidateBibinfoCache(). This doesn't prevent filesystem checks---we don't control whether bib files might have changed---but it allows insets to force invalidation of the cache and force a reload whenever the user might have changed the list of bibfiles, etc. Richard
RE: LyX 1.6.2: possible performance issue inside master document
>> Do we need a new class that we use internally. In this class we store >> a filename, which is case-sensitive on case-sensitive filesystem, that >> is guaranteed to be a long filename and not a short one (on Windows), >> that is not a symlink, that is relative or absolute (pick one or both). >> >> >This could work, though it looks as if we're duplicating what QFileInfo >does, and we might run into similar problems about caching to what we >had before. I don't know---but I think this makes some sense out of why >they want to return false if the files don't exist. The problem, as I see >it, is that QFileInfo objects are getting created so often, which requires >filesystem access, even independently of the caching stuff. And I'm not >sure how that can be avoided. Why is the class called FileName by the way ? This really sounds like a class that governs the FileNAME and all related things. And QFileInfo really sounds like things that have to do with the physical file. Anyway, I want to have a class which can freely be instantiated and another class with the warning that it can have performance issues because it's related to files on the disk. We might duplicate things, but with our own simple FileName class we are _sure_ that we don't access the disk. By the way, to convert filenames and stuff we still can use QFileInfo to be cross-platform. We just use it once, and when the filename is in our own class, we are sure that it has the correct format. That was the idea. >Anyway, here's another idea. Why are we calling loadIfNeeded() every time? True, that should be fixed too. >But it bothers me that we're checking whether every needed file is loaded >every time we type a key. Me too. >And, while we're at it, maybe a similar solution could work for updateMacros(). Yes, you might remember I was complaining about the infinite number of calls to updateMacros a while ago. If you think about it, why would we want to call the updateMacros function of a child document. Like that could have changed by pressing a character in the master document ?!? I hope other devs also have an opinion on this, but I guess we should fix it all of the above. >Richard Vincent
Re: LyX 1.6.2: possible performance issue inside master document
Hi all, thank Abdelrazak Younes for the prompt answer. so I'm not the only one (see users list: Lyx 1.6.2 unusably slow on the Mac) Child document opened alone or together with its parent? Performance inside the child document is much better. Only the master is concerned. It does not matter whether the master is opened or not. It even does not make any difference if the master or the child is opened first. For me it looks like heavy internal updating inside the master is done for each included child-doc (it does not matter whether the child is opend). If you are able to send me a test document (maybe privately) I'll have a look. Even better: put it in a new bugzilla entry. Mailing is a problem since it's a whole project documentation, with pictures ... Hopefully I can reproduce it with smaller documents. But I think some othe guys already have the same trouble (see above). I'm not familar with bugzilla, but if I can reproduce it I'll give it a try. Thank's Zardoz -- Aufgepasst: Sind Ihre Daten beim Online-Banking auch optimal geschützt? Jetzt absichern: https://homebanking.gmx.net/?mc=m...@footer.hb
Re: LyX 1.6.2: possible performance issue inside master document
pseudo2...@gmx.at wrote: Child document opened alone or together with its parent? Performance inside the child document is much better. Only the master is concerned. It does not matter whether the master is opened or not. It even does not make any difference if the master or the child is opened first. For me it looks like heavy internal updating inside the master is done for each included child-doc (it does not matter whether the child is opend). If you are able to send me a test document (maybe privately) I'll have a look. Even better: put it in a new bugzilla entry. Mailing is a problem since it's a whole project documentation, with pictures ... Hopefully I can reproduce it with smaller documents. But I think some othe guys already have the same trouble (see above). I'm not familar with bugzilla, but if I can reproduce it I'll give it a try. I wish I had some idea why this is happening only on certain platforms. Or is it only on certain platforms? I've not seen reports of this behavior under Linux, but maybe there are people with similar problems. Have there been other changes to the Windows or Mac distributions from 1.6.1 to 1.6.2, besides the changes made internally to LyX? Richard
