Bo Peng wrote:
Dear all,
Two approaches to embed or bundle external files have been presented
in the 'alternative to individual embedding?' thread. If you are
interested, but did not have time to follow the heated discussions,
here is a summary.
Here's my summary.
As far as I can see, there is only one real difference between the
approaches, at the level of design:
(i) Bo allows "selective embedding", meaning you can choose which files
to embed and which not to embed. I, on the other hand, have just two
states: embed all; embed none. Bo has described use cases in which one
would want the former. I agree that such cases exist, but don't actually
think anyone would use LyX that way. And I think that the costs, both in
terms of code complexity and in terms of UI clutter, aren't worth it.
The main use here involves bundling/embedding everything the LyX file needs.
Bo also emphasizes this one:
(ii) Bo packs everything into the LyX file using base64 encoding; I put
the files in a subdirectory and then wrap the whole bundle into a zip file.
But that's just a confusion, and there's no real difference here. First:
I OPTIONALLY wrap the bundle into a single file. There is a real
difference between a "bundle"---that's just a LyX file with an
associated directory---and what we might call a "wrapped bundle", which
is the single file. Second: How we wrap the bundle is an implementation
detail. If we want to wrap it by base64 encoding everything, then we can
do that. Or we can use tar. Or whatever.
So there's really only one thing to discuss. That said, I'll try to
clear up some confusions below.
Approach one: individual embedding.
1) Each and every external file can be embedded individually, using an
embed checkbox in related inset dialogs.
2) The embedded files are written directly in the '.lyx' file encoded in base64.
3) If users provide a password, the original filenames of embedded
files are saved in .lyx file. Otherwise, the information is lost.
4) Global 'embed-all' and inset-level features (undecided yet) such as
'un-embed', 'update from external file' will be provided. These
features can make use of file link information if they are available.
For example, 'extract all embedded files' will be able to recover the
.lyx file and external files in their original directory structure.
This achieves full reversibility between 'embed all' and 'extract all
embedded files', which are called 'bundle' and 'unbundle' in the
second approach.
Major benefits:
1) There is no change to existing .lyx files. The .lyx format is still
in plain text.
2) Users can work with external, potentially out of tree, files as before.
3) Users can embed/un-embed individual inset, bundle/unbundle all
external files, without loss of information.
The criticisms:
1) These 'embed' checkboxes cluster the inset dialogs.
2) The .lyx format is more difficult to work with than a zip format
(approach two). Base64 strings may confuses index-generators such as
google desktop.
3) The code needed for this approach is much more complex than the code
needed for the other approach.
My answers:
1) Individual embedding has its own advantages. There is no
alternative because this approach is characterized by not having a
global bundled mode.
This doesn't seem like much of an answer. It seems like conceding the point.
2) The plain text file is more svn friendly, easier to work with for
tasks other than extracting external files. I do not really care about
index-generation applications. I also pointed out that this format is
more natural for individual embedding. (No zip/unzip states).
This is just confused, because approach (2) does not use a zip format.
I've explained this above.
Overall, I think this approach is not intrusive, and without any major
shortcoming.
Approach two:
1) A 'bundled mode' is introduced. File format and working style are
preserved in 'unbundled mode'.
2) In the bundled mode, all external files are copied to a
subdirectory filename.lyxdir.
3) Switching from unbundled to bundled mode will move all external
files to this subdirectory.
It will COPY all external files to this directory.
4) Switching from bundled mode to unbundled mode will not move any
file. It merely stops copying files to this directory.
5) Original filenames are stored in a session file. The original user
on the original machine can 'update from external' to update the
embedded version from the external version. This update process may be
automated in some way.
6) If compression is turned on, filename.lyxdir will be zipped to filename.lyx.
It gets wrapped to filename.lyz, and the tool used to wrap it is an
implementation detail.
Major benefits:
1) No change to .lyx file format (but with permanent changes to the
content of a .lyx file when switching to bundled mode)
2) The zip format is easier to work with when lyx is not available.
Criticisms:
1) Not friendly to existing files. A .lyx file will be permanently
changed if it is turned to the bundled mode. Switching back to
unbundled mode will not recover the original .lyx file. (Richard said
that he does not offer reversibility in this sense.)
OK, now I understand what some of this reversibility talk has been
about. I thought we were talking about restoring the bundled files, not
restoring the LyX file. So I haven't said I don't offer reversibility in
that sense, since I wasn't ever talking about that. And I don't say
that. I could do reversibility in this sense if I wanted to do so, and
in much the same way Bo does.
If I understand things correctly, in Bo's case, the LyX file is changed
when a file is embedded, only not (he says) "permanently". The original
path is not kept in the \begin_inset bit of the LyX file for security
reasons; rather, a mapping between the embedded files and their original
locations is maintained, and that can then be used to restore the LyX
file. (I take it that this would happen when one turned off embedding
for the individual file?) But doing this just involves using the
mapping, so we can do it as long as we have the mappoing, and I have it,
too. The only difference here is that I have been thinking it should be
kept not in the LyX file but in the session file, but that isn't a real
difference either, since I can do it either way, too. So there's no real
difference here.
2) Troubles with content management systems such as subversion. Other
than having to reorganize existing files for existing documents
because of 1), users have to guess which files are added to
filename.lyxdir during editing to add them to subversion. This is
because lyx determines where to put them, under which name. (This is
troublesome to me, but maybe not to others)
This could be automated via the world's simplest shell script.
3) Difficult to work with external files. In the bundled mode, all
external files are bundled so users have to rely on (auto) update from
external file to make their changes to external files available to
lyx. Once a file is bundled, there is no way to unbundle it because
there is no individual bundling. Switching to unbundle mode will not
unbundle the file, this is potentially very confusing.
As (again) I've said several times, there is no real difference here
either. On Bo's approach, unembedding a file means copying it to an
empty subdirectory. On my approach, the file simply lives in
filename.lyxdir/, and you can do as you wish with it. If I just copied
filename.lyxdir/ to emptydir/, then I'd have EXACTLY what you get from
Bo's "unembed all files". But that does seem rather pointless.
4) Can not maintain meaningful external file structure. External files
are copied to filename.lyxdir in a predefined structure with possible
aliased names to avoid name conflict. It is argued that users can keep
their chapter1/figure1.png chapter2/blah structure for 'update from
external' purposes, but this option is only meaningful to the orignal
author. Also, once these external files are removed (they are not part
of the document), the structure is lost.
I disagree with this, but I don't have time to disentangle all the
confusions here. I've got final exams to grade.
rh
