Re: [zfs-discuss] A versioning FS

[Chad Leigh -- Shire.Net LLC]

On Oct 6, 2006, at 7:33 PM, Erik Trimble wrote:

David Dyer-Bennet wrote:

On 10/6/06, Nicolas Williams <[EMAIL PROTECTED]> wrote:

> >Maybe Erik would find it confusing.  I know I would find it
> >_annoying_.
>
> Then leave it set to 1 version

Per-directory?  Per-filesystem?

Whatever.  What's the actual issue here?

I don't recall that on TOPS-20 it was possible to not version.  What
you could do is set your logout.cmd file to purge your space down to
one copy when you logged out.

But see, that assumes you have a logout-type functionality to use. Which indeed is possible for command-line usage, but then only in a very limited way. During a typical session, I access almost 20 NFS-mounted directories. And anyone using autofs/automount trees gets even more. You're saying that my logout script has to know about all of them to keep things clean? That's unrealistic.

It is up to you to come up with a scheme to keep things clean, the same way you do now anyway (downloads, etc),

And that still doesn't solve the problem of people who use SAMBA or NFS from machines which don't have an interactive shell logout system (i.e. Windows).

No real issue.

It is still mounted on their desktops and they can still delete files with FV the same way they do now

This worked fine for the users I knew; even on a system that didn't
have as much as a gigabyte of disk storage total to support a few
dozen software engineers.

The problem is we are comparing apples to oranges in user bases here. TOPS-20 systems had a couple of dozen users (or, at most, a few hundred). VMS only slightly more. UNIX/POSIX systems have 10s of thousands.

Rarely.  Most of them have in the same range as VMS now or then.

Plus, the number of files being created under typical modern systems is at least two (and probably three or four) orders of magnitude greater. I've got 100,000 files under /usr in Solaris,

so?  You are not editing these are you?

and almost 1,000 under my home directory.

again, FV only matters when you edit them

And I don't have anything significant in my /home (no source code, no build/test trees, just misc business stuff). What is managable with a few files quickly becomes unwieldy with more than a few dozen.

I think you admitted you had not used FV before. Is that the case? Then how can you speak about what becomes unwieldy? FV is not any more unwieldy with 1000 files in a dir than with 10. Most people are not editing the 1000 files sitting in their directory.

This is what Nico and I are talking about: if you turn on file versioning automatically (even for just a directory, and not a whole filesystem), the number of files being created explodes geometrically.

Again, it does not.  Files are only versioned when they are edited.

> The above should be simple to do however -- a program does an open of > a file name "foo.bar". ZFS / the file system routine would use the

> most recent version by default if no version info is given.

How can version information be given without changing the APIs or
putting the version number/string into the file name?

about.  I'd find it useless without that; it has to be a real part of
the filesystem, usable by everybody, not a special addon accessible
only with one or two dedicated applications.

The version number is part of the file name in all the examples I know

Putting the version number/string into the file name is hard for me to

accept.  It's what would lead to polluting my directories.

Set your ls default to not show versions.  Isn't the problem then

solved? Maybe add that option to the GUI filesystem explorer as well.

But this requires modifying all the relevant apps, which is the same amount of work as modifying them to use a new FV API. It's not transparent to the end-user.

ie,

"foo" gets you the latest "foo"



No app changes are necessary.

Because the semantics of a file name are different on a unix/posix system than they are on a VMS or TOPS-20 system, which had more structured filenames. I would say that the version cannot be an actual part of the file name but would have to be meta data. However, it could display as part of the username and the underlying system can be made to do the right thing Specifically entering in foo;7 gets you version 7 or the latest if there are less than 7 versions available. The app can think of it as being part of the file name, but the underlying system would have to know how to do the right thing in extracting the version out and making it meta data. Takes some thinking and I am not claiming to have all the answers right now, but hardly undoable.

In practice, it never was a problem that I noticed, or that other
people noticed.  And remember that this was on slower systems with
smaller screens and often rather slower screen update.

Do you not like the idea based on theory, or did you actually use
TOPS-20 for a while and find the versioning troublesome?

Putting the file version number as part of the file name breaks things. Apps unaware of the special significance of this format will tend to write similar names, which can screw everything royally.

Example:

Say we use <file>;<version>

In emacs, I edit FOO:2

it will write out a temp file "FOO:2~". So, how does the FS deal with this the next time they need to create a new version? The problem lies in that under VMS, the ';' was a special character, and unusable in normal naming. I suspect a similar situation exists under TOPS-20. No such luck in a POSIX filesystem - all printable (and many unprintable) characters are valid for use in filenames. So you _CAN'T_ use them to deliniate File Versioning, without risking blowing the entire scheme when some random app decides to either use your FV marker for its own needs, or something similar to the emacs case above.

Yes, this needs to be thought about but is hardly a show stopper. There are most likely many possible solutions that will work for most people, and if you make it configurable then those people who run into issues can reconfigure it. Ie, say you do use <file>;<version> and that proves unworkable in a specific case, then the system can be reconfigured to display / decode using a different character. Or perhaps, in that case, the user needs to supply a \ character in front of the ; that exists in a real file to not have it decoded as a version identifier.

> one UI is the command line shell

Indeed!  And command-line tools, like ls(1), find(1), etc...


my files without "echo *" or "ls" showing them to me by default.

What I'm saying is that I'd like to be able to keep multiple versions of

And I find that completely unacceptable; useless.  The whole point of
putting versioning in the filesystem is that that makes it accessible
to all programs.

But, because of the explosion in the number of files,

There is no explosion. You have not made any case except your claim in mail that such an explosion is real and not just your personal fear. Remember, only files that are edited/changed are FVed.

you CAN'T automatically show all versions.

Sure you can.

Users will NEVER accept this.

Have you done usability testing? There is no explosion of files like you claim so most users would probably not object.

The only clean way to do this is to show file versions only upon request. Not by default.

Your claim. I claim otherwise. You'd have to do some real testing to see if it really is a problem.

> >What if an application deals in multiple files?
>
> so?

So, file versions aren't useful unless the application explicitly
decides tells the OS when to make them.

File versions are created when a file is created.  In the scenario

the old file is kept and the new file is given the version number +1
of the old file.

where, today, an existing file would be overwritten (deleted), instead

Exactly

Similarly with applications that keep files open but keep writing

app.  E.g., databases.  fsync(2) helps here, but lots and lots of
fsync(2)s would result in no useful versioning.

transactions in ways that the OS can't isolate without input from the

None of those are candidates for file versioning, and a darned good thing, too.

Honestly, as far as file versioning goes, the time to make a new version is when calling open() with the appropriate arguments to allow for append or modification.

exactly

You obviously don't want to create a new version if you are only opening a file for read-only access, and changing version on fsync () is ludicrous,

yes

and on close() doesn't differentiate between a file which has been modified or not.

This would have to be thought out some.

ok. I am not an expert on low level file operations so I don't know what knowledge if around of a file having been changed or not. However, I'd have to think back to my VMS dev days -- I think that when using the LSE editor, whenever I did a write it did create a new version. I cannot remember for sure. Would have to find a VMS system to test on.

Given this, we're back into the problem FV is supposed to solve. It is entirely possible for an editor to keep open a file for a long time, periodically writing out your changes without issuing a new open(). Word with auto-save turned off is a prime example.

ok

Given this, you've only created a new version when you first load the document, and all your intermediary changes are lost, since it only saves the document on close().

ok. FV is not a panacea to all problems. But most people do not sit there with a file open forever. FV solves a lot of problems. It still acts as a checkpoint for file edits -- especially for most people's standard usage of open a file, edit it, close it, go watch TV, think of something else, come back in and open again and edit it, etc.

Thus, in order to get benefits from FV, your editor must issue periodic close() and open() commands on the same file, as you edit, all without your intervention.

No, you get the benefits of FV, just across editing sessions and not internal to an editing session.

Exactly how many editors do this? I have no idea. So, the only way to enable FV is to require the user to periodically push the "Save" button. Which is how much more different than the current situation?

Chad

I edit a file. I realize I screwed up. I can go back to the previous version (or 2 ago or whatever). I cannot do that in the current situation.

-Erik


_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

---
Chad Leigh -- Shire.Net LLC
Your Web App and Email hosting provider
chad at shire.net

smime.p7s
Description: S/MIME cryptographic signature

_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss