Re: Server-side SavePoints [was: Subversion Design Contribution Question]

2017-11-02 Thread Nathan Hartman 
On Nov 2, 2017, at 9:49 PM, Daniel J. Lacks, PhD  wrote:
> 
> Hi Julian,
> 
> I appreciate your insight into thinking about the problem and your knowledge 
> of SVN is evident.  You are correct about my intentions, while it is possible 
> for the user to reuse commands (or scripts) to do such an operation (local or 
> remote), I was hoping that reuse made the SVN software development of a 
> capability easier to implement as an SVN command.  I am not against 
> client-side shelving or checkpoints, but I think that a server-side 
> capability is also useful.  I am in favor of both client-side AND server-side 
> stashing concepts, not necessarily only doing one or the other.  

Hi all,

I've been busy and haven't chimed in for some time about SavePoints (which were 
still called check points when I last participated in the discussion). First 
I'd like to say that I like the name SavePoints and agree with the rationale 
stated at the wiki, particularly that with this name, the command can be 
shortened to sp and that it will likely make more sense to non-native speakers 
of English than check points.

More to the point of this thread, I think the idea of client- and server-side 
SavePoints does make sense for the reasons stated previously. I like very much 
the idea that server side SavePoints could provide some of the underlying 
machinery for code review. I'd like to add the following to the list of 
potential use cases: Given the core svn library / third party value-added 
nature of Subversion, I can imagine a feature in, say, TortoiseSVN, where 
SavePoints are taken locally immediately, and when a line of communication to 
the server exists (which could be at the same time or at some later time) the 
SavePoints would be automatically transferred (copied / synched) to the server. 
That is, SavePoints are local and provide the ability to "work offline" for 
extended periods, but transparently become "online" at the next opportunity 
when communication permits. One could say that local SavePoints are 
transparently backed up to the server.

I think this would give users some very good benefits. For one, it would reduce 
or eliminate the need to remember to transfer SavePoints to the server 
manually. It would provide a safety net in case of loss of a WC, as occurs when 
a virtual machine goes haywire and the user deletes it with all its content and 
replaces it with a fresh VM and new checkout. (This has happened to me enough 
times!) It also answers a concern I have with anything that is local as opposed 
to server-side: my working copies are all in a RAM-disk! It speeds up builds 
and searches considerably, reduces wear on my laptop's flash storage, and 
enforces in my mind the idea that the working copy is very temporary and can be 
lost or damaged at any time--which encourages small frequent commits. So with 
the addition of local SavePoints, the concern is that if I'm in a RAM-based 
working copy, the SavePoints are no more durable than the working copy itself. 
But if the SavePoints are synchronized to the server, either manually or 
automatically / transparently, then I can continue to work happily in my 
RAM-disk while reaping the benefits of SavePoints and the safety of centralized 
version control.

Hopefully these thoughts are beneficial to the community.

Kind regards,
Nathan

Re: Server-side SavePoints [was: Subversion Design Contribution Question]

2017-11-02 Thread Daniel J. Lacks, PhD 

Julian,

No problem, I thought your responses were helpful and constructive.  I 
am interested to help, at this point I want to know what is the best way 
for me to do that.  For example, how to start: brainstorming, 
collaborating (hosting a meeting, chat channel, or opening an issue 
ticket), or just do it?   What format: Wiki, Word document, PowerPoint 
presentation, etc.?  How much research should I do: you mentioned 
client-side tool integration (TortoiseSVN, etc.) and other tools for 
comparison (Git, Perforce, etc.)?  My assumption is you are perhaps 
looking to take a step back from what is already posted about SavePoints 
and perhaps address a design that includes client side and server side 
use cases, scenarios, and approaches.  Then at some point I assume it is 
reviewed and versioned, then someone can decide what to implement now or 
in the future.


Feel free to contact me offline if that makes sense, I am new to this.  
I want to help, but I also want to get it right the first time to save 
everyone's time.


Thanks,

Danny


On 11/1/2017 9:25 AM, Julian Foad wrote:

Julian Foad wrote:

Have we thought of the server-side possibility? Of course. [...]
I realized what I said can come across as rather critical and 
dismissive of the idea. I hope that you or others will be equally 
critical of my work. My start on shelving is nothing more than a thin 
wrapper around 'svn diff' and 'svn patch' at this stage, so the same 
questions apply to it. I think how it works is the simple UI gives me 
a mental framework that helps me think about the potential use cases 
and further requirements, whereas using diff and patch directly would 
tend to constrain my thinking to the use cases and limitations of 
those existing commands.


So another way I could respond is, sure, that may be the start of 
something good and useful, so please go ahead and start creating it 
and/or thinking further and expanding on the details. Others have 
asked for this server-side kind of shelving before, and Perforce 
shelving is server-side. Maybe only by trying it will we really learn 
how useful it is and what for. Might you have the time to take the 
idea further? It would be great if you did; even a little input like 
your initial inquiry is helpful.


- Julian



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Server-side SavePoints [was: Subversion Design Contribution Question]

2017-11-02 Thread Daniel J. Lacks, PhD 

Hi Julian,

I appreciate your insight into thinking about the problem and your 
knowledge of SVN is evident.  You are correct about my intentions, while 
it is possible for the user to reuse commands (or scripts) to do such an 
operation (local or remote), I was hoping that reuse made the SVN 
software development of a capability easier to implement as an SVN 
command.  I am not against client-side shelving or checkpoints, but I 
think that a server-side capability is also useful.  I am in favor of 
both client-side AND server-side stashing concepts, not necessarily only 
doing one or the other.  Thus, having a command option like "svn shelve 
--remote" will automate the capability making the implementation 
consistent across SVN users, repos, projects, versions, operating 
systems, etc.  My focus is on usability for the person with a lot of 
work to do, and for getting a new developer with little or no SVN 
experience to learn the capability quickly and feel confident that it's 
going to work and not mess up their files when stashing.  Also, while 
Git stashing is very useful, I think server-side stashing is a variation 
on that capability which suits SVN's implementation as a client-server 
model.  I also really like the idea of using this capability in 
conjunction with the changelists, whether the changes are stashed 
locally or remotely.  And maybe changelists will magically handle 
directories in the process of implementing this since we will need to 
stash those too  .


I answered the questions below inline.

Thanks,

Danny


On 11/1/2017 6:36 AM, Julian Foad wrote:
Hello Daniel. Thanks for your interest in discussing this. Emailing 
this list is indeed the right way to do so.


Having answered that meta question, I am changing the email Subject to 
reflect the substantive topic.


Daniel J. Lacks, PhD wrote:

I am interested in the design topic for SavePoints. I was wondering

> if anyone considered server-side stashes instead of client-side?

I am currently working on Shelving and Checkpointing, features which 
are more or less what the SavePoints page envisions (although not 
necessarily the implementation it describes). References:


https://wiki.apache.org/subversion/SavePoints
https://issues.apache.org/jira/browse/SVN-3625 "Commit shelving"
https://issues.apache.org/jira/browse/SVN-3626 "Commit checkpointing"
>> Assuming I understand the shelving request correctly, regardless if 
the SavePoints are a new command or not, it looks like they want that 
ability to be part of the switch command.  I think that is brilliant 
from the ease of use perspective.
>> For checkpoints, I have encountered this type of desire myself to 
not want to make a commit but needing to backup my work.


>> In both of these cases and stashing/SavePoints capability in 
general, in order to achieve these capabilities now, I implement this in 
a script to see what files I change and move or back them up somewhere 
in my home directory with the same file structure as they exist in SVN.  
Sometimes I tar and zip the files up too, in the use case of 
checkpointing I suppose.  There certainly are manual or scripting 
workarounds for making this work, but a capability baked into SVN would 
make me feel more confident that I am not goofing something in my script 
that will erase my real changes because of a malformed mv command, or 
handling a special case like file permission problems (not that either 
of these have happened before (ok, maybe they did)).


These are deliberately client-side features for the main reasons of 
speed and offline usability. Speed is critical: one of the main use 
cases is to be able to switch *quickly* between small tasks, and this 
has to be useful for people using a relatively slow WAN connection.


>> I agree with the comments about speed being a concern for stashing, 
but SVN could cache remote stashes locally upon checkouts, switches, 
updates, etc. in the .svn folder and also makes implementation easier 
perhaps for client-side stashing in the process.  Thus, if someone 
does a local stash, the files are saved in the .svn directory, but 
perhaps are not sync'ed up to the server unless explicitly requested.



Have we thought of the server-side possibility? Of course. (See for 
example the 2nd/3rd comments in those issues.) But Subversion already 
supports saving work to a server-side branch, so we need to ask, what 
do you need for your use case, beyond the present ability to use 
branches? So let's explore further.
>> To answer the use case question (from my perspective off the top of 
my head):


1. Must be easy to use.  Whether the default is client-side or 
server-side or two different commands, make this something easy to pick 
up.  Also, combining the capability with other commands like merge, 
changelists, switches, etc. again makes this easy especially if there is 
a way to do it without learning new arguments.  Ease of use also comes 
into play, in my opinion, by offering the server

Re: Subversion Design Contribution Question

2017-11-02 Thread Daniel J. Lacks, PhD 

Sounds like a good idea Paul, something I did not think of.


Danny


On 11/1/2017 4:37 AM, Paul Hammant wrote:
I'm also interested in standardized server-side handling of such 
things for a different reason to the one you state - code reviews. 
Well outside the purview of vanilla Subversion for sure, but a feature 
that the portal vendors have or are coding themselves. I've a love of 
Trunk-Based Development and got to see the code review system that 
Google built for themselves around Perforce (Subversion was partially 
inspired by Perforce back in 2000). Developers at their workstations 
would declare 'done', and initiate code review. The changelist and 
some metainfo would be zipped up and pulled to somewhere central. The 
bulk of the workflow is in the UI Guido Van Rossum led at Google and 
showcases in https://www.youtube.com/watch?v=sMql3Di4Kgc (2006). Post 
code review (and CI added metrics) the change-list could be 
reconstituted and committed (submitted in Perforce lang).  It is the 
humble little save point that facilitates the 25,000 developers 
co-existing in one trunk with Piper (their 2012 replacement to 
Perforce) and ultimately bots integrating (Martin Fowler's preferred 
language for merge to trunk/master/mainline when practicing CI) change 
sets ever few seconds.


I'm much less interested in workflows where I'm sharing something by 
that mechanism for others to work on, as that's not Trunk-Based 
Development.


-ph




---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Error E160016 "... is not a directory in filesystem ..."

2017-11-02 Thread Vincent Lefevre 
On 2017-11-03 00:18:59 +0100, Vincent Lefevre wrote:
> cventin:~> svn ls -r99809 
> file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
> svn: E160016: Failure opening '/perso/tcl/fidelite'
> svn: E160016: '/perso/tcl' is not a directory in filesystem 
> '99759db8-4ec0-0310-8bf9-df86780d22d8'

For this one, valgrind doesn't signal any issue. And with strace,
I can see that:

.../db/revs/99/99809 is opened 5 times.
.../db/revs/99/99807 is opened 3 times.
.../db/revs/52/52653 is opened 5 times.

r99807 is the revision where the /perso/tcl directory and the
/perso/tcl/fidelite file were added.

r52653 is the revision where the /misc/perso directory was renamed
to /perso.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Error E160016 "... is not a directory in filesystem ..."

2017-11-02 Thread Vincent Lefevre 
[Moving to dev]

On 2017-11-01 17:05:24 +0100, Vincent Lefevre wrote:
> Also, this is not consistent, unless future commits have an influence
> on the following command:
> 
> joooj:~> svn ls -r99809 
> file:///srv/d_joooj/home/svn/root/perso/tcl/fidelite@99808
> svn: E160016: Failure opening '/perso/tcl/fidelite'
> svn: E160016: '/perso/tcl' is not a directory in filesystem 
> '99759db8-4ec0-0310-8bf9-df86780d22d8'
> 
> ioooi:~> svn ls -r99809 
> file:///srv/d_ioooi/home/svn/root/perso/tcl/fidelite@99808
> fidelite
> 
> FYI, ioooi is the old server, which I migrated with dump+load to
> joooj at revision r102683. So, I should have obtained the same
> result; the one given by ioooi is the expected one.

This is definitely a bug, and a regression in 1.9.

I did the following test:

1. On a Debian/unstable machine (which has Subversion 1.9.7 r1800392),
I created a format 3 FSFS repository and loaded the dump.

2. I tested the issue:

cventin:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

cventin:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

cventin:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
svn: E160016: Failure opening '/perso/tcl/fidelite'
svn: E160016: '/perso/tcl' is not a directory in filesystem 
'99759db8-4ec0-0310-8bf9-df86780d22d8'

cventin:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
svn: E160016: Failure opening '/perso/tcl/fidelite'
svn: E160016: '/perso/tcl' is not a directory in filesystem 
'99759db8-4ec0-0310-8bf9-df86780d22d8'

So, this is buggy.

3. I copied the repository to a Debian/jessie machine (which has
Subversion 1.8.10 r1615264).

4. I tested the issue on this machine:

tomate:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

tomate:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

tomate:~> svn ls -r99808 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99809
fidelite

tomate:~> svn ls -r99809 
file:///home/vlefevre/private/svn-tmp/perso/tcl/fidelite@99808
fidelite

Everything is fine with the old version.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)