Re: Keyword substitutions not being merged correctly
Guten Tag David Sandberg, am Mittwoch, 20. März 2013 um 18:17 schrieben Sie: > 4) Now, Sam merges trunk revision 200 into his feature branch. The > merge happens automatically without prompts for conflicts or > anything, and the resulting file A contains MOST of the changes from > the trunk revision ... EXCEPT for the Revision keyword, which stays at 101! Those keywords are substituted only by the client after the commit, there are not part of the merge and therefore the old values persist until a commit is made. During updates the keyword is updated because something has changed for this file, but again from the client and the keyword is not part of the data in the repo. > Because we have business logic that depends upon these revision > numbers being merged properly, this problem has a segment of our > company actively talking about switching back to VSS, for goodness > sake, so if anyone has a solution for this I would be very > appreciative. I think you misunderstood the concept of those keywords. > Several of the preceding descriptions use the phrase “last known” or > similar wording. Keep in mind that keyword expansion is a > client-side operation, and your client “knows” only about changes > that have occurred in the repository when you update your working > copy to include those changes. If you never update your working > copy, your keywords will never expand to different values even if > those versioned files are being changed regularly in the repository. http://svnbook.red-bean.com/en/1.7/svn.advanced.props.special.keywords.html Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
RE: Keyword substitutions not being merged correctly
Guten Tag David Sandberg, am Mittwoch, 20. März 2013 um 18:17 schrieben Sie: > > 4) Now, Sam merges trunk revision 200 into his feature branch. The > > merge happens automatically without prompts for conflicts or anything, > > and the resulting file A contains MOST of the changes from the trunk > > revision ... EXCEPT for the Revision keyword, which stays at 101! > > Those keywords are substituted only by the client after the commit, there are > not > part of the merge and therefore the old values persist until a commit is made. > During updates the keyword is updated because something has changed for this > file, > but again from the client and the keyword is not part of the data in the repo. The part of this I am not understanding is that the commit was made by the user Jim ... shouldn't that change be a part of the file thereafter? Because at that point I can look directly at the contents of the repository (via TSVN's Repo Browser) and see the updated revision number has been committed. It is after that point that said revision is merged by Sam into a branch, and the already-updated revision # gets lost. Or are you saying that, when I look at the repository contents afterwards, it is my own SVN client that is adding in the revision number for me at that point, by looking back to see what the most recent revision of that file is? Things I had read elsewhere suggested that the keyword substitution takes place on the client during the commit process, but is it actually the case that the client only updates the committing user's working copy of that file during the commit, and then similarly updates any other user's working copy of that file with the same revision number during an update? If this is the case, then am I on the right track by thinking that the revision # of the merged file wouldn't be updated because the client doesn't regard a merge as the same thing as an update, and the revision # would only be updated when that merged file is committed (and to the revision of the merge commit in that branch, not the revision of the original trunk modification)? Thank you for the reply, and for any additional clarification! (BTW, I have read the relevant section of the SVN book several times, and don't seem to be able to glean the answers to these behaviors from its contents.) - David Sandberg
Re: Keyword substitutions not being merged correctly
On 03/20/2013 02:24 PM, David Sandberg wrote: > Things I had read elsewhere suggested that the > keyword substitution takes place on the client during the commit process, > but is it actually the case that the client only updates the committing > user's working copy of that file during the commit, and then similarly > updates any other user's working copy of that file with the same revision > number during an update? If this is the case, then am I on the right > track by thinking that the revision # of the merged file wouldn't be > updated because the client doesn't regard a merge as the same thing as an > update, and the revision # would only be updated when that merged file is > committed (and to the revision of the merge commit in that branch, not > the revision of the original trunk modification)? Perhaps the most non-obvious-yet-useful thing folks need to understand about Subversion and keywords is this: the Subversion server knows *nothing* about keywords or the substitution thereof. When keyword substitution is enabled, the Subversion client will substitute keyword values into placeholder locations embedded the file when copying the working file out of the "pristines" area of the working copy administrative subdirectory. When you commit changes to such a file, the client first un-substitutes the keywords found therein -- leaving only the keyword placeholders -- before transmitting those file changes up to the server. If the commit is successful, the Subversion client will again re-substitute your keywords into the working file, this time with an updated revision and last-modified stuffs. Likewise, when you update the file to receive someone else's changes, the client re-substitutes the keywords in the working file with the updated metadata. This is why your user isn't seeing updated revisions in his keywords after performing a merge -- because the line which contains those keywords in the file didn't *really* change from the server's point of view. It only appears to change on the client side because the client keeps refreshing that line of the file when required by changes to the file's local version. You should find that after accepting the merge from the server, the keyword remains unchanged in your working copy, but that after committing the merged changes, the working copy file is updated to reflect the new version you've just committed. -- C. Michael Pilato CollabNet <> www.collab.net <> Enterprise Cloud Development signature.asc Description: OpenPGP digital signature
RE: Keyword substitutions not being merged correctly
> Perhaps the most non-obvious-yet-useful thing folks need to understand about > Subversion and keywords is this: the Subversion server knows *nothing* about > keywords or the substitution thereof. > > When keyword substitution is enabled, the Subversion client will substitute > keyword values into placeholder locations embedded the file when copying > the working file out of the "pristines" area of the working copy > administrative > subdirectory. When you commit changes to such a file, the client first > un-substitutes the keywords found therein -- leaving only the keyword > placeholders -- before transmitting those file changes up to the server. > If the commit is successful, the Subversion client will again re-substitute > your keywords into the working file, this time with an updated revision and > last-modified stuffs. Likewise, when you update the file to receive someone > else's changes, the client re-substitutes the keywords in the working file > with the updated metadata. > > This is why your user isn't seeing updated revisions in his keywords after > performing a merge -- because the line which contains those keywords in the > file didn't *really* change from the server's point of view. It only appears > to change on the client side because the client keeps refreshing that line > of the file when required by changes to the file's local version. > > You should find that after accepting the merge from the server, the keyword > remains unchanged in your working copy, but that after committing the merged > changes, the working copy file is updated to reflect the new version you've > just committed. Michael, Thank you for that crystal clear explanation, which accounts perfectly for the observed behavior. I will add that I am not sure I agree with the correctness of that behavior, but that's only because it doesn't happen to fit our operational model, in which we cherry-pick revisions from the trunk into our release branch and then build archives for deployment to customers. The problem is that the resulting deployed files need to reflect the revision numbers that were committed into the trunk ... or at least some newer revision number than in previous builds. From what you've described, we could perhaps achieve that by committing the merged set of files to our release branch before we begin a build, but that seems like a backwards way to go about things when we may find ourselves still needing to make updates as part of completing the build, so we would be committing potentially broken revisions in order to get these keyword substitutions to update before the build process runs. Anyway, at least now we know what is going on here, and can explore process Changes or workarounds for our needs. Thanks again! - David Sandberg
RE: Keyword substitutions not being merged correctly
> From: David Sandberg [mailto:david.sandb...@hickorytech.com] > Sent: Wednesday, March 20, 2013 1:18 PM > > 1) User Jim commits a new file A with the Revision keyword to the trunk > in revision 101. > 2) User Sam merges trunk revision 101 into his feature branch, and the > new file A comes across fine and shows revision 101 in the keyword > substitution part of the file. > 3) Jim commits an update to file A to the trunk, in revision 200. (If > other users who are working in the trunk pull this file at this time, > it updates and the keyword in their updated file correctly reads 200.) > 4) Now, Sam merges trunk revision 200 into his feature branch. The > merge happens automatically without prompts for conflicts or anything, > and the resulting file A contains MOST of the changes from the trunk > revision ... EXCEPT for the Revision keyword, which stays at 101! As Thorsten pointed out, the keywords do not store the actual revision number. Only the unexpanded "$Rev$" is stored on the server. Its sounds like "svn:keywords" isn't set correctly on Sam's feature branch. Whenever a new revision of a file is created (such as after a merge) the revision keyword should match the "Last Changed Rev:" (which you can see when you run "svn info" on the file.) What does Sam see when he runs "svn info" in step 4? After Sam's merge in step 4 (and before Sam's commit,) does a "svn diff" show any differences in the file? If the merge didn't actually change anything, then the file won't change, so the "Last Changed Rev:" doesn't change, so the revision keyword doesn't change. Also, you won't see the revision keyword change in the file until after you do a commit.
Re: Keyword substitutions not being merged correctly
On 03/20/2013 02:52 PM, David Sandberg wrote: > Thank you for that crystal clear explanation, which accounts perfectly for the > observed behavior. I will add that I am not sure I agree with the correctness > of that behavior, but that's only because it doesn't happen to fit our > operational model, in which we cherry-pick revisions from the trunk into our > release branch and then build archives for deployment to customers. The > problem > is that the resulting deployed files need to reflect the revision numbers that > were committed into the trunk ... or at least some newer revision number than > in previous builds. Yeah, so you've cherry-picked changes and applied them to some base revision. I think that, given some thought, you would agree that it would be incorrect for Subversion to claim that your working file is now "at" the revision you cherry-picked. I mean, to be "at" revision N implies that a file's history contains every change N, N-1, N-2 ... all the back to the file's origin. > From what you've described, we could perhaps achieve that > by committing the merged set of files to our release branch before we begin a > build, but that seems like a backwards way to go about things when we may find > ourselves still needing to make updates as part of completing the build, so we > would be committing potentially broken revisions in order to get these keyword > substitutions to update before the build process runs. Hrm. It seems to me you're placing far too much stock in the revision number, attributing descriptive powers to it for which it was never designed and is ill-suited to provide. Revision numbers alone cannot uniquely describe anything. You can have 15 files, each in one of 15 branches, each of which has slightly different file contents from the next ... and yet all 15 variations could carry the same $Revision$ keyword value (perhaps because all 15 copies were modified in the same revision). In such a scenario it becomes obvious that a revision number alone is insufficiently as a unique representation of that any one of those files. Rather, I would suggest that any attempt to uniquely identify a file's version include not just its revision number but also its path in the repository. path@rev is a truly unique coordinate in the Subversion repository. That might mean employing unique tags for your deployed configurations which are created directly from your cherry-pick-carrying branch working copies. $ svn checkout BRANCH_URL branch-wc $ svn merge [WHATEVER] SOURCE_URL branch-wc $ svn copy branch-wc ^/tags/MY_CUSTOM_TAG \ -m "Tag baseline XYZ + cherry-picked revisions N, M, P, and Q" Given such tags and the use therein of, say, the $URL$ keyword (instead of, or in addition to, the $Revision$ one), uniqueness can be achieved. -- C. Michael Pilato CollabNet <> www.collab.net <> Enterprise Cloud Development signature.asc Description: OpenPGP digital signature
RE: Keyword substitutions not being merged correctly
> As Thorsten pointed out, the keywords do not store the actual revision number. > Only the unexpanded "$Rev$" is stored on the server. > > Its sounds like "svn:keywords" isn't set correctly on Sam's feature branch. It wasn't, in fact, but after Michael's response I realize that it wouldn't have mattered even if it had been set, because we were relying upon that value to be set prior to any commit of the merged file into the branch. But it is a good reminder that, if we do change our operational pattern to commit merges before we build, we also need to make sure the keywords and auto-prop settings are set properly on that system. > Also, you won't see the revision keyword change in the file until after you > do a commit. Yes, this was apparently the crux of the issue. Thanks for the response! - David Sandberg
Re: Keyword substitutions not being merged correctly
Guten Tag David Sandberg, am Mittwoch, 20. März 2013 um 19:24 schrieben Sie: > The part of this I am not understanding is that the commit was made > by the user Jim ... shouldn't that change be a part of the file > thereafter? No, because the number is introduced by the client only, the keyword is in the repo. > Or are you saying that, when I look at the repository contents > afterwards, it is my own SVN client that is adding in the revision > number for me at that point, by looking back to see what the most > recent revision of that file is? Exactly that. > Things I had read elsewhere > suggested that the keyword substitution takes place on the client > during the commit process, ...and updates. > but is it actually the case that the > client only updates the committing user's working copy of that file > during the commit, and then similarly updates any other user's > working copy of that file with the same revision number during an > update? Yes. > If this is the case, then am I on the right track by > thinking that the revision # of the merged file wouldn't be updated > because the client doesn't regard a merge as the same thing as an > update, and the revision # would only be updated when that merged > file is committed (and to the revision of the merge commit in that > branch, not the revision of the original trunk modification)? Yes. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
RE: Keyword substitutions not being merged correctly
> Yeah, so you've cherry-picked changes and applied them to some base revision. > I think that, given some thought, you would agree that it would be incorrect > for Subversion to claim that your working file is now "at" the revision you > cherry-picked. I mean, to be "at" revision N implies that a file's history > contains every change N, N-1, N-2 ... all the back to the file's origin. That's a very good point. We have never done cherry-picking of these particular files, so the revision # could be trusted a bit more, but if we did, I can see how issues could result, and I'll be bringing that up with the other stakeholders in this issue. Thanks much for that. > It seems to me you're placing far too much stock in the revision number, > attributing descriptive powers to it for which it was never designed and is > ill-suited to provide. > > Revision numbers alone cannot uniquely describe anything. You can have 15 > files, > each in one of 15 branches, each of which has slightly different file contents > from the next ... and yet all 15 variations could carry the same $Revision$ > keyword value (perhaps because all 15 copies were modified in the same > revision). > In such a scenario it becomes obvious that a revision number alone is > insufficiently as a unique representation of that any one of those files. In our case, that level of descriptiveness is not really vital ... all we need is to know the filename we are working on (during the deploy) and a revision number that we can compare to other revision numbers from that same file, in order to determine if the file is newer that anything that had previously been deployed to the server in question (because we store those earlier revision numbers on a file by file basis). Including the path wouldn't seem to help with making that determination. But returning to your earlier point, if we had ever had a scenario where we had cherry-picked revisions 1 and 3 for one of these files, and then later brought in revision 2, the fact that (in our presumption) the revision # in the earlier deployed file would already have been #3 would have caused our scheme to break down. It is a great point, as I said, and might be sufficient to motivate us to do something different. I appreciate all of the very useful input. - David Sandberg
Re: Keyword substitutions not being merged correctly
On Wed, Mar 20, 2013 at 2:07 PM, C. Michael Pilato wrote: >> > Revision numbers alone cannot uniquely describe anything. You can have 15 > files, each in one of 15 branches, each of which has slightly different file > contents from the next ... and yet all 15 variations could carry the same > $Revision$ keyword value (perhaps because all 15 copies were modified in the > same revision). In such a scenario it becomes obvious that a revision > number alone is insufficiently as a unique representation of that any one of > those files. The revision numbers are global, so wouldn't it take some bizarre mucc command or a commit from a level above the branches to change the contents of those files to different things in the same commit? I guess it would be possible, but it would take a very odd workflow. -- Les Mikesell lesmikes...@gmail.com
RE: Keyword substitutions not being merged correctly
> -Original Message- > From: David Sandberg [mailto:david.sandb...@hickorytech.com] > Sent: Wednesday, March 20, 2013 2:52 PM > > Thank you for that crystal clear explanation, which accounts perfectly > for the observed behavior. I will add that I am not sure I agree with > the correctness of that behavior, but that's only because it doesn't > happen to fit our operational model, in which we cherry-pick revisions > from the trunk into our release branch and then build archives for > deployment to customers. The problem is that the resulting deployed > files need to reflect the revision numbers that were committed into the > trunk ... or at least some newer revision number than in previous > builds. From what you've described, we could perhaps achieve that by > committing the merged set of files to our release branch before we > begin a build, but that seems like a backwards way to go about things > when we may find ourselves still needing to make updates as part of > completing the build, so we would be committing potentially broken > revisions in order to get these keyword substitutions to update before > the build process runs. That's why we grab the 'Last Changed Rev:" of the root directory of the release tree and use that as the release number. The root revision is guaranteed to change if there's a change anywhere in its subtree. In my experience, trying to manipulate meta-data inside of the files is just not worth the effort except in one-off script files. Meaning, why are you storing information *about* the file *inside* of the file? (And even in the case of one-off files, I'm more interested in the svn URL than the actual revision number.) IMO, you would skip a lot of angst if you just update the keywords during your release packaging process, i.e. have your build/packaging script walk the files and update the "keywords" manually. Even so, keywords don't really provide much value, in that they don't tell you what the correct revision should be (i.e. is the correct release installed?) nor do they reveal file tampering (i.e. production changes.) IMO, you need to adopt a proper installer script and create an audit script. (Gnu tar has a --diff option, which gives you quick and dirty packaging, deployment, and auditing functionality.) Once you have those in place, the "need" for revision keywords goes away.
