Author: danielsh
Date: Thu Mar 24 23:37:13 2022
New Revision: 1899186
URL: http://svn.apache.org/viewvc?rev=1899186&view=rev
Log:
[in site/staging]
Add a rendered version of the issue #525 user guide, maintained in markdown on
the pristines-on-demand-on-mwf branch, so we can point people not to the
markdown source but to the rendered HTML.
* docs/i525-user-guide.html: Rendered version of i525-user-guide.md.
See keyword expansions within.
.
Please don't set svn:keywords on this file :)
* docs/release-notes/1.15.html
(#bare-working-copies): Point to it.
Added:
subversion/site/staging/docs/i525-user-guide.html (with props)
Modified:
subversion/site/staging/docs/release-notes/1.15.html
Added: subversion/site/staging/docs/i525-user-guide.html
URL:
http://svn.apache.org/viewvc/subversion/site/staging/docs/i525-user-guide.html?rev=1899186&view=auto
==============================================================================
--- subversion/site/staging/docs/i525-user-guide.html (added)
+++ subversion/site/staging/docs/i525-user-guide.html Thu Mar 24 23:37:13 2022
@@ -0,0 +1,368 @@
+<p>This is a detailed user guide to the "i525pod" feature.</p>
+
+<p>$LastChangedDate: 2022-03-24 23:26:24 +0000 (Thu, 24 Mar 2022) $</p>
+
+<p>$LastChangedRevision: 1899185 $</p>
+
+<p>$URL:
https://svn.apache.org/repos/asf/subversion/branches/pristines-on-demand-on-mwf/notes/i525/i525-user-guide.md
$</p>
+
+<h1>Terminology</h1>
+
+<p>Place-holders:</p>
+
+<ul>
+<li>"i525pod" stands for the name of the feature documented here, that is
+as implemented on branch 'pristines-on-demand-on-mwf' on 2022-03-08.
+(Not any other interpretation of what Issue #525 discusses.)</li>
+<li>"bare" stands for the state of a WC in which the feature "i525pod" is
+enabled, and so contains only some of the pristine copies.</li>
+</ul>
+
+<p>Terminology:</p>
+
+<ul>
+<li>"pristine copy" or "pristine" or "text base": a copy of a file's content
+matching the corresponding base revision in the repository. For any file
+type, not necessarily text format. Enables e.g. local diff and revert,
+and delta update and commit. Stored in the WC metadata area. The term
+herein refers only to file content, although Subversion also stores the
+pristine copy of properties and of tree structure.</li>
+<li>"hydrate" a pristine copy: to fetch the pristine copy from the
+repository and store it in the WC metadata area.</li>
+<li>"dehydrate" a pristine copy: to remove the pristine copy from the WC
+metadata area, while remembering that it may be needed again.</li>
+<li>"sync scope": the set of WC paths in which a Subversion operation will
+check for pristines that need to be hydrated or dehydrated. This is a
+superset of the pristines that the operation will actually need.</li>
+<li>"operation": a high level Subversion operation, such as "diff" or
+"merge" or "update"; e.g. a subcommand of the 'svn' program.</li>
+</ul>
+
+<h1>"i525pod" User Guide</h1>
+
+<h2>Functional and Timing Differences</h2>
+
+<p>This section details the functional and timing differences when the
+"i525pod" feature is used.</p>
+
+<p>In a WC where "i525pod" is enabled (see other sections for how), basic usage
+differs from that found in previous versions of Subversion (1.14 in case of
+doubt) in the following ways.</p>
+
+<p>Each of the following operations, that previously were <em>offline</em>
operations,
+will now contact the repository to "hydrate" pristine copies before
+beginning its function, if (and only if) any pristines within the "sync scope"
+are found to be locally modified and currently "dehydrated".</p>
+
+<ul>
+<li><code>svn cat</code> (default case: base version)</li>
+<li><code>svn diff</code> (default case: base against working)</li>
+<li><code>svn resolve</code> (also conflicts resolver in <code>merge</code>,
<code>update</code>)</li>
+<li><code>svn revert</code></li>
+</ul>
+
+<p>Notes on previously <em>offline</em> operations,:</p>
+
+<ul>
+<li>Contacting the repository may require authentication.</li>
+<li>If contact or authorization fails, the operation will error out and not
+be available.</li>
+<li>This contact may be needed as a result of a file being modified that is
+not of interest in the current operation, as the "sync scope" is a
+superset of the pristines that this operation will actually need.</li>
+<li>The "hydrating" phase may take a long time, and (currently) gives no
+progress feedback, before the operation begins its usual (previous)
+behaviour.</li>
+</ul>
+
+<p>[TODO: update that if we add progress feedback]</p>
+
+<p>Each of the following operations, that previously were <em>online</em>
operations,
+also will now require the same.</p>
+
+<ul>
+<li><code>svn diff</code> (comparing repository to WC)</li>
+<li><code>svn merge</code></li>
+<li><code>svn switch</code></li>
+<li><code>svn update</code></li>
+<li><code>svn checkout --force</code> (similar to update)</li>
+</ul>
+
+<p>Additional notes on previously <em>online</em> operations:</p>
+
+<ul>
+<li>The "hydrating" phase requires its own connection and authentication to
+the repository, which is not [currently] shared with the main part of
+the operation. This may mean a password would have to be entered an
+additional time, for example, depending on the configuration.</li>
+<li>In some edge cases, there may be some difference in the outcome of the
+operation. A possible example is if repository path authorization has
+been withdrawn from a path that now needs to be hydrated; this
+particular case is still under discussion in issue #????.</li>
+</ul>
+
+<p>[TODO: check/eliminate the additional authentication? Issue filed?]</p>
+
+<h2>Disk Space Usage Differences</h2>
+
+<p>This section details the disk space usage differences when the feature is
+used.</p>
+
+<p>Summary:</p>
+
+<ul>
+<li>Without "i525pod" feature, a Subversion WC typically occupies
+approximately twice the size of the working files, because there is a
+pristine copy of every unique working file, plus some other metadata.</li>
+<li>The "i525pod" feature enables a WC to occupy a disk space little greater
+than the size of the working files, in cases where only a small
+proportion (size wise) of the files are locally modified at any one
+time.</li>
+<li>Cases where the feature will not provide much disk space benefit
+include:
+<ul>
+<li>where a large proportion of WC files are duplicates (because there was
+only one pristine copy in the first place for each set of duplicates);</li>
+<li>where the WC is arranged to contain only (predominantly) the files
+that are to be modified in a given work flow.</li>
+</ul></li>
+</ul>
+
+<p>Initial WC size, from checkout/upgrade:</p>
+
+<ul>
+<li>on a fresh checkout, no pristines are stored;</li>
+<li>on enabling the feature in an existing WC (by WC upgrade), pristines are
+removed for unmodified files;</li>
+<li>[TODO] check: are pristines removed in fact for all files not just
+unmodified files?</li>
+</ul>
+
+<p>Size increase and decrease:</p>
+
+<ul>
+<li>the operations listed in the 'Functional differences' section may grow
+and/or shrink the disk space used, as detailed there;</li>
+<li>operations such as editing a file (without Subversion's control), and
+editing its Subversion properties, will not cause any change to the
+pristine storage;</li>
+<li>'commit' will remove the pristines for files it commits that had locally
+modified content;</li>
+</ul>
+
+<p>Disk Full:</p>
+
+<p>Failure modes when disk becomes full during an operation...</p>
+
+<ul>
+<li>[TODO] Investigate.</li>
+</ul>
+
+<h2>Support, Compatibility, Enabling</h2>
+
+<p>In brief:</p>
+
+<p>"i525pod" is an optional feature in Subversion 1.15. It can be enabled
+separately for each WC. By default "i525pod" is not enabled and WCs remain
+compatible with Subversion 1.8 through 1.14. To enable "i525pod" for a given
+WC, check out or upgrade the WC to 1.15's format. A WC in
+1.15's format is no longer compatible with older Subversion
+clients.</p>
+
+<p>For details: see "Working Copy Format Upgrade and Compatibility"
section.</p>
+
+<p>[TODO: update if/when we use a specific feature-enable flag.]</p>
+
+<h2>User Guide: Principle Of Operation</h2>
+
+<p>Quoting the original 'BRANCH-README':</p>
+
+<p>"The core idea is that we start to maintain the following invariant: only
+ the modified files have their pristine text-base files available on the
+ disk."</p>
+
+<p>When "i525pod" is enabled in a given WC, Subversion stores pristine copies
+according to the following principle:</p>
+
+<ul>
+<li>It stores the pristine copy of each file that is currently in a locally
+modified state.</li>
+<li>It does not store the pristine copy of each file that is not currently
+in a locally modified state.</li>
+<li>At certain points in its operations, Subversion checks the modified
+state of certain files, and fetches (from the repository) or removes
+their pristine copy accordingly if the state has changed.</li>
+</ul>
+
+<p>Subversion updates the pristine storage to match this principle at certain
+times.
+ - To get into the appropriate state at the beginning of the operation, we
+ walk through the current text-base info in the db and check if the
+ corresponding working files are modified. The missing text-bases are
+ fetched using the 'svn_ra' layer. The operations also include a final
+ step during which the no longer required text-bases are removed from
+ disk.</p>
+
+<ul>
+<li>The operations that don't need to access the text-bases (such as "svn
+ls" or the updated "svn st") do not perform this walk and do not
+synchronize the text-base state.</li>
+</ul>
+
+<p>Subversion updates the pristine storage to match this principle at certain
+times.</p>
+
+<ul>
+<li>At the beginning of any high level operation that may need to access
+pristine copies [1], Subversion first checks for files [1] that are now
+locally modified and whose pristine copies are not already present, and
+connects to the repository and fetches them.</li>
+<li>At the beginning and/or the end of those operations, Subversion checks
+for files that are now unmodified, and removes their pristine copies.
+That includes files that became unmodified before the operation, and,
+for some operations (such as commit) also files that became unmodified
+because of the action of the operation.</li>
+<li>It does not matter how a file became modified or unmodified: whether a
+Subversion operation caused that to be the case, or whether the user had
+externally edited the file into that state some time before.</li>
+<li>[1] See "Which operations?"</li>
+<li>[2] See "Which files?" about the "sync scope".</li>
+</ul>
+
+<p>The pristine storage state does not immediately change to match the
+principle:</p>
+
+<ul>
+<li>NOT whenever the user edits a file outside of Subversion's control,</li>
+<li>NOR for files outside the "sync scope" of a given operation,</li>
+<li>NOR during any Subversion operation other than those listed.</li>
+</ul>
+
+<p>The definition of "locally modified" takes into account Subversion's local
+"translation" options such as "keywords" and "eol-style". A working file
+that differs from the repository copy only in keywords and/or EOL-style is
+not regarded as locally modified. When Subversion needs to access a pristine
+copy of such a file, Subversion makes a temporary pristine copy by
+"detranslating" the working file. It may store this in temporary disk space
+for the duration of the operation, but does not keep this indefinitely.</p>
+
+<h3>Which operations will "hydrate" or "dehydrate" pristines?</h3>
+
+<p>The operations deemed to need the pristine copies of locally modified files,
+and which therefore "hydrate" (and "dehydrate") the pristine copies of
+locally modified files, are:</p>
+
+<ul>
+<li><code>H</code> <code>D</code> <code>svn cat</code> (default case: base
version)</li>
+<li><code>-</code> <code>D</code> <code>svn commit</code> (dehydrate only)</li>
+<li><code>H</code> <code>D</code> <code>svn diff</code> (default case: base
against working)</li>
+<li><code>H</code> <code>D</code> <code>svn resolve</code> (also conflicts
resolver in <code>merge</code>, <code>update</code>)</li>
+<li><code>H</code> <code>D</code> <code>svn revert</code></li>
+<li><code>H</code> <code>D</code> <code>svn switch</code></li>
+<li><code>H</code> <code>D</code> <code>svn update</code></li>
+<li><code>-</code> <code>D</code> <code>svn upgrade
--compatible-version=1.15</code> (upgrade to 1.15's WC format enables
"i525pod")</li>
+</ul>
+
+<h3>Which files does Subversion "hydrate" or "dehydrate" ("sync scope")?</h3>
+
+<p>Which files does Subversion "hydrate" or "dehydrate", whenever an eligible
+operation has the chance to do so?</p>
+
+<p>The files that a given operation "hydrates" and/or "dehydrates" are neither
+all possible files in the working copy, nor the minimal subset necessary for
+the particular operation.</p>
+
+<p>Not maximal:</p>
+
+<ul>
+<li>It considers files to "hydrate" or "dehydrate" within one or more
+sub-trees that encompass all the target paths passed to the operation.
+Depending on the operation, this sub-tree may span anything from the
+whole WC (typical for "update", for example), right down to one specific
+file of interest, or even a directory containing no files at all.</li>
+</ul>
+
+<p>Not minimal:</p>
+
+<ul>
+<li>The operation in the end may not look at all the files in "sync scope":
+for example, because of filtering options (such as <code>--depth</code>,
+<code>--changelist</code>), or because the operation terminated early (for
+example, <code>svn resolve</code>... and choose <code>quit</code>).</li>
+<li>The operation in the end may not read the pristine copy of every file it
+processes: for example, <code>svn diff --properties-only</code>.</li>
+</ul>
+
+<h1>Working Copy Format Upgrade and Compatibility</h1>
+
+<h2>Summary</h2>
+
+<ul>
+<li>"i525pod" is active when a given WC is in 1.15-compatible format.</li>
+<li>To use "i525pod", use <code>svn checkout --compatible-version=1.15</code>
+or <code>svn upgrade --compatible-version=1.15</code>.</li>
+<li>Subversion 1.15 by default uses 1.8-compatible WC format, with "i525pod"
+inactive in those WCs, the same as Subversion 1.8 through 1.14.</li>
+<li>Subversion 1.14 and older cannot read or write a 1.15-compatible WC.</li>
+<li>Working copies cannot be downgraded</li>
+</ul>
+
+<h2>Details</h2>
+
+<p>To use "i525pod" in a given working copy (WC), that WC's metadata needs to
be in a new 1.15-compatible format (also called WC format 32). You need a
Subversion 1.15 client to create or use a WC in this format.</p>
+
+<p>You can either check out a working copy in this format:</p>
+
+<pre><code>svn checkout --compatible-version=1.15
+</code></pre>
+
+<p>or upgrade an existing working copy from a 1.8-compatible or older format
to this format:</p>
+
+<pre><code>svn upgrade --compatible-version=1.15
+</code></pre>
+
+<p>That working copy:</p>
+
+<ul>
+<li>becomes "bare" right away;</li>
+<li>is no longer accessible by Subversion clients below version 1.15;</li>
+<li>cannot be converted back to 1.8-compatible format.</li>
+</ul>
+
+<p>[TODO] We might change this so that upgrading to 1.15-compatible format and
enabling "i525pod" are separate steps and the latter is optional.</p>
+
+<p>Subversion 1.15 also supports, and uses by default, the working copy format
that has been in use since Subversion 1.8 (format 31). A working copy in that
format does not support "i525pod" and is never "bare".</p>
+
+<pre><code>svn checkout --compatible-version=1.8
+svn checkout # the same, as 1.8 is the default
+</code></pre>
+
+<p>You do not need to use the new format for all your working copies.
Subversion 1.15 can work with some working copies in 1.8-compatible format and
others in 1.15-compatible format.</p>
+
+<h2>Upgrade / Downgrade</h2>
+
+<p>You can upgrade to 1.15-compatible format (WC format 32) with:</p>
+
+<pre><code>svn upgrade --compatible-version=1.15
+</code></pre>
+
+<p>from either</p>
+
+<ul>
+<li>1.8-compatible format, which Subversion 1.15 can use; or</li>
+<li>1.7-compatible or older format, which Subversion 1.15 cannot use but can
upgrade.</li>
+</ul>
+
+<p>By default "svn upgrade" upgrades a working copy to 1.8-compatible format.
This is useful for upgrading a WC from the 1.7 and older formats so Subversion
1.8 and newer can use it.</p>
+
+<pre><code>svn upgrade
+svn upgrade --compatible-version=1.8 # the same
+</code></pre>
+
+<p>You CANNOT downgrade any working copy to an older format.</p>
+
+<p>(If you need a working copy in an older format than your current Subversion
client supports, you would have to check out a working copy using an older
Subversion client that supports the format you want to use.)</p>
+
+<h2>[TODO] Enable/disable "i525pod" in a 1.15-compatible WC</h2>
+
+<p>[TODO] Not yet implemented (2022-03-08).</p>
Propchange: subversion/site/staging/docs/i525-user-guide.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: subversion/site/staging/docs/i525-user-guide.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Modified: subversion/site/staging/docs/release-notes/1.15.html
URL:
http://svn.apache.org/viewvc/subversion/site/staging/docs/release-notes/1.15.html?rev=1899186&r1=1899185&r2=1899186&view=diff
==============================================================================
--- subversion/site/staging/docs/release-notes/1.15.html (original)
+++ subversion/site/staging/docs/release-notes/1.15.html Thu Mar 24 23:37:13
2022
@@ -199,7 +199,8 @@ users. We'll cover those in this sectio
<p class='todo'>Point to the "User Guide" in <a
href="https://svn.apache.org/repos/asf/subversion/branches/pristines-on-demand-on-mwf/notes/i525/";
-><tt>notes/i525/</tt></a>; ideally, to an HTML-rendered version thereof</p>
+><tt>notes/i525/</tt></a>. Regenerate <a href="/docs/i525-user-guide.html"
+><tt>/docs/i525-user-guide.html</tt></a> from the source markdown.</p>
<p>All Subversion working copies require extra storage space in addition to
the size of the checked out files.</p>
