AW: copying subdirectories in subversion 1.7

[Markus Schaber]

Hi, Daniel,

Von: Daniel Shahaf [mailto:d...@daniel.shahaf.name]

> Markus Schaber wrote on Mon, Jul 25, 2011 at 15:57:00 +0200:
> > Hi, Stefan,
> >
> > Von: Stefan Sperling [mailto:s...@elego.de]
> > > > So I'm not only "detaching" subdirectories, but also
"re-implanting"
> > > > them afterwards.
> > >
> > > Sounds like what you really need is this as-of-yet not implemented
> > > feature:
> > > http://subversion.tigris.org/issues/show_bug.cgi?id=3625
> >
> > Yes, that one would definitely help. :-)
> >
> > Maybe extending "svn diff" and "svn patch" to preserve
> > copy/rename/history information would already be half of that
> > implementation? A "shelve" subdir under .svn/ could then be used to
> > store the diff files.
> >
> 
> That's one solution.
> 
> However, wc.db already has the ability to represent multiple related
trees
> (via op_depth), so another way would be to extend that into storing
not
> only the post-'svn patch' trees too, and then merging them in the
usual
> way.  (which may require repository communication, or storing the pre-
> patch tree locally too...)

I think your suggestion sounds a little bit more complex to start with.

Shelving of patches in .svn/shelve/ can be implemented independently of
the diff/patch extension (with the disadvantage of losing history for
now), and can even be implemented as an external tool / script.

Extending the "diff" and "patch" commands to preserve history (maybe
just by a pointer to url@rev) has benefits for other uses (like sending
patches to someone else).

> Perhaps you'd be interested in contributing to the implementation of
this
> feature?

Yes, I am, but I've only very limited time to work on it while being at
home.


Best regards,

Markus Schaber

Re: AW: copying subdirectories in subversion 1.7

[Daniel Shahaf]

I think you've made your point by now.  However, 1.7 is close enough to
being released that this feature will NOT be included in it.  How do you
suggest to proceed?

Personally, I'd suggest documenting this in the 1.7 release notes and
contributing the hands to work on this for 1.8.

Les Mikesell wrote on Mon, Jul 25, 2011 at 10:13:43 -0500:
> I'm surprised

Re: AW: copying subdirectories in subversion 1.7

[Stefan Sperling]

On Mon, Jul 25, 2011 at 10:13:43AM -0500, Les Mikesell wrote:
> On 7/25/2011 9:58 AM, Mark Phippard wrote:
> >You agree but you are also missing or glossing over my point.  When you
> >copied that subdirectory you were not using a feature of SVN, you used
> >the OS.  SVN cannot provide an option on the OS copy command.
> 
> And you seem to have missed the point that people understand and use
> OS level commands and expect them to work.  I'm sure I didn't invent
> the idea of doing that, so it has almost certainly been promoted as
> an advantage of the design on this list and in usage tutorials if
> not in official documentation.

As I've already mentioned, to the best of my knowledge, copying subdirs
to create new working copies was never recommended in official documentation.
 
> >Given how closely you follow the project, I am surprised you are
> >surprised.
> 
> I'm not surprised that the capability is not there in the
> new/different WC format.  I'm surprised that there is no option to
> maintain currently-expected behavior in a release version.

It is wrong to expect it. It doesn't even work properly in 1.6 in
some cases (you cannot copy a locally added tree without its parent
and expect it to work as an independent working copy -- not that
such an operation would make sense).

> >This release has taken over 2 years and the new WC design
> >has not changed from the original proposal.  If you go back to the lists
> >you will see the ramifications of this change were being discussed even
> >while we were still working on SVN 1.6 and the need for an svn detach
> >was raised back then.
> 
> Which makes it even more surprising that it was omitted.

You cannot omit something which doesn't exist yet.

The 'svn detach' subcommand has not been added because there was no time
left to implement it in the 1.7.x release cycle. There are enough
improvements in 1.7.x that make the upgrade worthwhile regardless.

Re: AW: copying subdirectories in subversion 1.7

[Stefan Sperling]

On Mon, Jul 25, 2011 at 09:49:13AM -0500, Les Mikesell wrote:
> I'm not arguing that the change is bad or shouldn't have been done,
> just that it is a very surprising change in design philosophy, and
> projects that make surprising design changes without concern for
> existing use patterns

And the fact that we carefully considered this case, discussed it, and
came to a conclusion, does not comfort you? That we documented known
workarounds for this uses case even though it has never been officially
supported? It's not like we are being secretive about it or didn't
care about users who use this "feature".

> make me nervous about what other surprises may be lurking in them.

Any other surprises we know about are documented in the release notes.
If there are additional ones we don't know about them and they will
hopefully be found during beta testing. If you find one, let us know :)

Re: AW: copying subdirectories in subversion 1.7

[Les Mikesell]

On 7/25/2011 9:58 AM, Mark Phippard wrote:


Agreed, but it is a logical design that falls out of the way
subdirectories work and the way people use them.


There are plenty of instances
where people did not want this behavior, and for them it is now
fixed.


Agreed again.  Although the change I would have preferred would have
been an option to not keep a pristine copy at all for the
circumstances where it doesn't work out well.


You agree but you are also missing or glossing over my point.  When you
copied that subdirectory you were not using a feature of SVN, you used
the OS.  SVN cannot provide an option on the OS copy command.


And you seem to have missed the point that people understand and use OS 
level commands and expect them to work.  I'm sure I didn't invent the 
idea of doing that, so it has almost certainly been promoted as an 
advantage of the design on this list and in usage tutorials if not in 
official documentation.



I'm not arguing that the change is bad or shouldn't have been done,
just that it is a very surprising change in design philosophy, and
projects that make surprising design changes without concern for
existing use patterns make me nervous about what other surprises may
be lurking in them.


Given how closely you follow the project, I am surprised you are
surprised.


I'm not surprised that the capability is not there in the new/different 
WC format.  I'm surprised that there is no option to maintain 
currently-expected behavior in a release version.



This release has taken over 2 years and the new WC design
has not changed from the original proposal.  If you go back to the lists
you will see the ramifications of this change were being discussed even
while we were still working on SVN 1.6 and the need for an svn detach
was raised back then.


Which makes it even more surprising that it was omitted.


The bottom line is that this has been a long
release cycle and the software is now good enough to use and benefit
from.  There is no reason to hold it back.


If surprising behavior doesn't bother you...


If being able to copy
subdirectories is important than stick with 1.6 or help to drive the
effort to get scripts or patches submitted so that the feature can make
it into 1.8.  If it is done with scripts, it can be delivered immediately.


I mostly use CentOS or RHEL, so if past history is any guide, I'll be on 
1.6 there for a long time...


--
  Les Mikesell
lesmikes...@gmail.com

Re: AW: copying subdirectories in subversion 1.7

[Mark Phippard]

On Mon, Jul 25, 2011 at 10:49 AM, Les Mikesell wrote:

> On 7/25/2011 9:24 AM, Mark Phippard wrote:
>
>>
>> In SVN 1.6 and earlier it is not like someone wrote code specifically to
>> allow people to copy/move folders out of their WC.  It was just
>> something that fell out of the design.
>>
>
> Agreed, but it is a logical design that falls out of the way subdirectories
> work and the way people use them.
>
>
>  There are plenty of instances
>> where people did not want this behavior, and for them it is now fixed.
>>
>
> Agreed again.  Although the change I would have preferred would have been
> an option to not keep a pristine copy at all for the circumstances where it
> doesn't work out well.


You agree but you are also missing or glossing over my point.  When you
copied that subdirectory you were not using a feature of SVN, you used the
OS.  SVN cannot provide an option on the OS copy command.  So there is no
way a user could copy a folder using the OS and not copy the .svn folder.
 My point was simply that SVN prior to 1.7 does not have a command for doing
what you are doing.  It just fell out of how the underlying OS worked.  That
is largely why SVN 1.7 still does not have such a command.  It is something
new that someone has to create and figure out how it should work etc. and no
one stepped up to do it.  If users really want it, someone will step up and
do it.


>> The opinion was that if users really want to be able to easily detach
>> folders from their working copy someone will step up with scripts and/or
>> patches to add a feature to SVN to do it.
>>
>
> I'm not arguing that the change is bad or shouldn't have been done, just
> that it is a very surprising change in design philosophy, and projects that
> make surprising design changes without concern for existing use patterns
> make me nervous about what other surprises may be lurking in them.
>

Given how closely you follow the project, I am surprised you are surprised.
 This release has taken over 2 years and the new WC design has not changed
from the original proposal.  If you go back to the lists you will see the
ramifications of this change were being discussed even while we were still
working on SVN 1.6 and the need for an svn detach was raised back then.  The
bottom line is that this has been a long release cycle and the software is
now good enough to use and benefit from.  There is no reason to hold it
back.  If being able to copy subdirectories is important than stick with 1.6
or help to drive the effort to get scripts or patches submitted so that the
feature can make it into 1.8.  If it is done with scripts, it can be
delivered immediately.


-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: AW: copying subdirectories in subversion 1.7

[Les Mikesell]

On 7/25/2011 9:24 AM, Mark Phippard wrote:



Von: Les Mikesell [mailto:lesmikes...@gmail.com
]

On 7/25/11 3:22 AM, Stefan Sperling wrote:

I am not aware of any existing developer who expressed
interest in
working on 'svn detach'.


Just curious: what drove the original WC design where each

subdirectory

was inherently independent?  And is this the same set of
developers?


My guess is that this Design was "inherited" from CVS.

And AFAICS, the initial set of developers had some common
members with
the CVS developers, and it changed gradually, there was no
"revolution"
replacing all existing developers with new ones.


The tradeoffs are sort of obvious, but it seems like a very drastic
change to make with no option to maintain the old behavior.


In SVN 1.6 and earlier it is not like someone wrote code specifically to
allow people to copy/move folders out of their WC.  It was just
something that fell out of the design.


Agreed, but it is a logical design that falls out of the way 
subdirectories work and the way people use them.



There are plenty of instances
where people did not want this behavior, and for them it is now fixed.


Agreed again.  Although the change I would have preferred would have 
been an option to not keep a pristine copy at all for the circumstances 
where it doesn't work out well.



For example, in Eclipse this has always been a huge problem.  Users
would copy and paste a folder and this would silently bring the .svn
folder with it which was not what was desired.  I mention Eclipse simply
because this is one of the only events where it does not provide a hook
for plugins to get involved so the SVN plugins for Eclipse are not able
to detect and fix this situation and it is perceived as a bug.

The opinion was that if users really want to be able to easily detach
folders from their working copy someone will step up with scripts and/or
patches to add a feature to SVN to do it.


I'm not arguing that the change is bad or shouldn't have been done, just 
that it is a very surprising change in design philosophy, and projects 
that make surprising design changes without concern for existing use 
patterns make me nervous about what other surprises may be lurking in them.


--
  Les Mikesell
   lesmikes...@gmail.com

Re: AW: copying subdirectories in subversion 1.7

[Mark Phippard]

On Mon, Jul 25, 2011 at 9:08 AM, Les Mikesell  wrote:

> On 7/25/11 7:49 AM, Markus Schaber wrote:
>
>>
>> Von: Les Mikesell [mailto:lesmikes...@gmail.com]
>>
>>> On 7/25/11 3:22 AM, Stefan Sperling wrote:
>>>
 I am not aware of any existing developer who expressed interest in
 working on 'svn detach'.

>>>
>>> Just curious: what drove the original WC design where each
>>>
>> subdirectory
>>
>>> was inherently independent?  And is this the same set of developers?
>>>
>>
>> My guess is that this Design was "inherited" from CVS.
>>
>> And AFAICS, the initial set of developers had some common members with
>> the CVS developers, and it changed gradually, there was no "revolution"
>> replacing all existing developers with new ones.
>>
>
> The tradeoffs are sort of obvious, but it seems like a very drastic change
> to make with no option to maintain the old behavior.


In SVN 1.6 and earlier it is not like someone wrote code specifically to
allow people to copy/move folders out of their WC.  It was just something
that fell out of the design.  There are plenty of instances where people did
not want this behavior, and for them it is now fixed.  For example, in
Eclipse this has always been a huge problem.  Users would copy and paste a
folder and this would silently bring the .svn folder with it which was not
what was desired.  I mention Eclipse simply because this is one of the only
events where it does not provide a hook for plugins to get involved so the
SVN plugins for Eclipse are not able to detect and fix this situation and it
is perceived as a bug.

The opinion was that if users really want to be able to easily detach
folders from their working copy someone will step up with scripts and/or
patches to add a feature to SVN to do it.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

AW: copying subdirectories in subversion 1.7

[Markus Schaber]

Hi, Stefan,

Von: Stefan Sperling [mailto:s...@elego.de]
> > So I'm not only "detaching" subdirectories, but also "re-implanting"
> > them afterwards.
> 
> Sounds like what you really need is this as-of-yet not implemented
> feature:
> http://subversion.tigris.org/issues/show_bug.cgi?id=3625

Yes, that one would definitely help. :-)

Maybe extending "svn diff" and "svn patch" to preserve
copy/rename/history information would already be half of that
implementation? A "shelve" subdir under .svn/ could then be used to
store the diff files.

Best regards

Markus Schaber

Re: AW: copying subdirectories in subversion 1.7

[Les Mikesell]

On 7/25/11 7:49 AM, Markus Schaber wrote:


Von: Les Mikesell [mailto:lesmikes...@gmail.com]

On 7/25/11 3:22 AM, Stefan Sperling wrote:

I am not aware of any existing developer who expressed interest in
working on 'svn detach'.


Just curious: what drove the original WC design where each

subdirectory

was inherently independent?  And is this the same set of developers?


My guess is that this Design was "inherited" from CVS.

And AFAICS, the initial set of developers had some common members with
the CVS developers, and it changed gradually, there was no "revolution"
replacing all existing developers with new ones.


The tradeoffs are sort of obvious, but it seems like a very drastic change to 
make with no option to maintain the old behavior.


--
  Les Mikesell
   lesmikes...@gmail.com

AW: copying subdirectories in subversion 1.7

[Markus Schaber]

Hi,

Von: Stefan Sperling [mailto:s...@elego.de]
> > > This works fine even for new directories, deleted ones, etc.
> > > Caveat: It cannot do copies yet -- those will show up as simple
adds.
> >
> > So moves and renames will lose their history?
> 
> Yes. This is because the patch format cannot represent copies and
moves.
> But how often do you split out a subtree that has copied or moved
items
> within it? Can't you commit such changes from the existing working
copy
> subtree before splitting it off?

My most common (ab-)usage is to use this as a poor man's local branch
feature. I'm working on some long-term task (e. G. some large
refactoring or new feature), and get interrupted by some short fix which
has to be developed on the fast lane. Often enough, my current working
copy state is not stable enough (or does not even compile), or the fix
touches the same files, but must be backportable independently, so I
have to create that fix on a clean code base.

Now, instead of checking out a new copy of the whole working environment
(or copying the working copy, which needs a similarly long amount of
time as we talk about about 2.5 Gigabytes in 58k files in about 38k
folders), I just copy the one or two interesting subdirectories into a
"safe" place, and locally revert all changes. After finishing and
committing the intermediate fix, I delete the subdirectories, move my
"safe" copies back into place, and then do "svn update".

So I'm not only "detaching" subdirectories, but also "re-implanting"
them afterwards.

With svn 1.7, this would require at least one additional step
("downgrade" the working copy to the previous revision before the
re-implant) and completely lose history for copies and renames.

And especially those larger tasks are those which tend to contain copies
and renames more often, as well as they have a higher likelihood to
collide with "fast-lane" fixes simply due to the fact that they need a
longer time.

Best Regards,
Markus Schaber

AW: copying subdirectories in subversion 1.7

[Markus Schaber]

Hi,


Von: Les Mikesell [mailto:lesmikes...@gmail.com]
> On 7/25/11 3:22 AM, Stefan Sperling wrote:
> > I am not aware of any existing developer who expressed interest in
> > working on 'svn detach'.
> 
> Just curious: what drove the original WC design where each
subdirectory
> was inherently independent?  And is this the same set of developers?

My guess is that this Design was "inherited" from CVS.

And AFAICS, the initial set of developers had some common members with
the CVS developers, and it changed gradually, there was no "revolution"
replacing all existing developers with new ones.

Best regards

Markus Schaber

___
We software Automation.

3S-Smart Software Solutions GmbH
Markus Schaber | Developer
Memminger Str. 151 | 87439 Kempten | Germany | Tel. +49-831-54031-0 |
Fax +49-831-54031-50

Email: m.scha...@3s-software.com | Web: http://www.3s-software.com 
CoDeSys internet forum: http://forum.3s-software.com
Download CoDeSys sample projects:
http://www.3s-software.com/index.shtml?sample_projects

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner |
Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915 

AW: copying subdirectories in subversion 1.7

[Markus Schaber]

Hi, Stefan,

> Von: Stefan Sperling [mailto:s...@elego.de]

> On Mon, Jul 25, 2011 at 10:37:36AM +0200, Markus Schaber wrote:
> > Hi, Stefan,
> >
> > Von: Stefan Sperling [mailto:s...@elego.de]
> > >> [Detaching of subdirectories of working copies]
> > > Honestly, we simply don't know how many people need this.
> >
> > Strictly speaking, it is not "needed" by anyone - one can always use
a
> > fresh checkout, and then manually merge all the changes.
> 
> 1.7 has 'svn patch' so transferring local changes to a new checkout
can be
> done in an automated way.
> 
>  cd ~/wc/subtree
>  svn diff > /tmp/subtree-changes.diff
>  cd ~
>  svn co $URL/subtree other_wc
>  svn patch /tmp/subtree-changes.diff other_wc
> 
> This works fine even for new directories, deleted ones, etc.
> Caveat: It cannot do copies yet -- those will show up as simple adds.

So moves and renames will lose their history?

> > But in my experience, a lot of users (including me) actually made
use
> > of the "copying a subdir just works" pattern.
> 
> Good to know.

My experiences cover several companies. In all of them, I had coworkers
using this pattern (as well as myself).

This is not representative, however.


Best regards

Markus Schaber

___
We software Automation.

3S-Smart Software Solutions GmbH
Markus Schaber | Developer
Memminger Str. 151 | 87439 Kempten | Germany | Tel. +49-831-54031-0 |
Fax +49-831-54031-50

Email: m.scha...@3s-software.com | Web: http://www.3s-software.com 
CoDeSys internet forum: http://forum.3s-software.com
Download CoDeSys sample projects:
http://www.3s-software.com/index.shtml?sample_projects

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner |
Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915