Problem with external file
I have a problem with external files on Windows XP using TortoiseSVN 1.6.99, Build 20232 - 32 Bit -dev, 2010/10/03 22:28:49. This is a nightly build which uses the WC 1.7 format. I reported it on the TortoiseSVN forum and they suggested reporting it here. Since this uses unreleased code, it was recommended that I send this to dev@ rather than us...@. To show the problem: (1) Create a project with an external file. (2) Modify and Commit a change to an actual working copy of the file (in another project). (3) Update the original project with the external file - even when omitting externals. This gives the following Subversion Exception: --- Subversion Exception! --- Subversion encountered a serious problem. Please take the time to report this on the Subversion mailing list (us...@subversion.apache.org) with as much information as possible about what you were trying to do. But please first search the mailing list archives for the error message to avoid reporting the same problem repeatedly. You can find the mailing list archives at http://subversion.apache.org/mailing-lists.html Subversion reported the following: In file 'C:\Users\kueng\nightlybuilds\latest\TortoiseSVN\ext\subversion\subversion\libsvn_wc\update_editor.c' line 4603: assertion failed (status == svn_wc__db_status_added) --- OK --- By the way, if I include externals in the Update in step (3) above, I get the same error. Also, if I make a mod to the external file in the original project and Commit the change, I get a similar problem. However, a complete new Checkout of the project with the external file works correctly, with the file changes being correctly incorporated. I am certain I did not have this problem in the past, but it may have been with the previous format when there were separate .svn folders for each folder in the project. Bob
Re: svn commit: r1005065 - in /subversion/branches/gpg-agent-password-store: ./ build/generator/ subversion/include/ subversion/include/private/ subversion/libsvn_auth_gpg_agent/ subversion/libsvn_sub
[Stefan Sperling] > > + /* Create the CACHE_ID which will be generated based on REALMSTRING > > similar > > + to other password caching mechanisms. */ > > + digest = svn_checksum_create(svn_checksum_md5, pool); > > + svn_checksum(&digest, svn_checksum_md5, realmstring, strlen(realmstring), > > + pool); > > Maybe use SHA1 instead? Why? All you're really after here is a unique key in a trusted datastore. Peter
Re: Subversion 1.6.13 Released
I actually got a notification it didn't get through, it looks like this: Subject: failure notice Hi. This is the qmail-send program at apache.org. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : Must be sent from an @apache.org address. I don't have any apache.org account so if you could send it it would be great Thanks /David On 2010-10-06 21:20, Daniel Shahaf wrote: This didn't get to announce@ because it wasn't sent from a *...@apache.org address. (I think the latter point bit cmpilato and hwright too; apparently, there's no notification to the sender in that case.) David, do you have an apache.org account? If not, I suppose one of us could re-send it, with From: set to ourselves and Sender: set to you, that might work? Thanks for the contribution! Daniel (a moderator of announce@) David Darj wrote on Wed, Oct 06, 2010 at 20:14:08 +0200: I'm happy to announce my release of Subversion 1.6.13 Win32 binaries and installer They are available at my website:http://alagazam.net and also on SourceForge: http://sourceforge.net/projects/win32svn/ Release notes for the 1.6.x release series may be found at: http://subversion.apache.org/docs/release-notes/1.6.html You can find the list of changes between 1.6.13 and earlier versions at: http://svn.apache.org/repos/asf/subversion/tags/1.6.13/CHANGES Questions, comments, and bug reports to us...@subversion.apache.org. Regards, David Darj http://alagazam.net
Re: Format 20 upgrade to NODES
Daniel Shahaf writes: > Okay, but I can do the f16->f18 upgrade using head of trunk, right? Yes, as far as I know. None of that code has changed. -- Philip
Re: Format 20 upgrade to NODES
Philip Martin wrote on Wed, Oct 06, 2010 at 20:31:56 +0100: > Daniel Shahaf writes: > > >> Greg Stein writes: > >> > >> > What about upgrades from f10 or f19? > >> > >> Upgrading from 1.6 works just as well with NODES as with BASE/WORKING, > >> we write NODES with op_depth 0 or 2. Ugrading from wcng 19 to 20 is > >> very simple, we just copy into NODES with op_depth 0 or 2. > > > > What about older f11-f18 wc's? On Monday I ran into an f16 wc. > > Nothing should have changed. They will auto-upgrade to f18 (you need > to do each directory as it hits the error), the single-db upgrade to > 19 must be done by a script, and then it will auto-upgrade to 20. > Okay, but I can do the f16->f18 upgrade using head of trunk, right? Or do I need to keep an SVN_WC__VERSION=18 build around? > -- > Philip
Re: Format 20 upgrade to NODES
Daniel Shahaf writes: >> Greg Stein writes: >> >> > What about upgrades from f10 or f19? >> >> Upgrading from 1.6 works just as well with NODES as with BASE/WORKING, >> we write NODES with op_depth 0 or 2. Ugrading from wcng 19 to 20 is >> very simple, we just copy into NODES with op_depth 0 or 2. > > What about older f11-f18 wc's? On Monday I ran into an f16 wc. Nothing should have changed. They will auto-upgrade to f18 (you need to do each directory as it hits the error), the single-db upgrade to 19 must be done by a script, and then it will auto-upgrade to 20. -- Philip
Re: Format 20 upgrade to NODES
Philip Martin wrote on Wed, Oct 06, 2010 at 19:00:58 +0100: > Greg Stein writes: > > > What about upgrades from f10 or f19? > > Upgrading from 1.6 works just as well with NODES as with BASE/WORKING, > we write NODES with op_depth 0 or 2. Ugrading from wcng 19 to 20 is > very simple, we just copy into NODES with op_depth 0 or 2. What about older f11-f18 wc's? On Monday I ran into an f16 wc.
Re: Subversion 1.6.13 Released
This didn't get to announce@ because it wasn't sent from a *...@apache.org address. (I think the latter point bit cmpilato and hwright too; apparently, there's no notification to the sender in that case.) David, do you have an apache.org account? If not, I suppose one of us could re-send it, with From: set to ourselves and Sender: set to you, that might work? Thanks for the contribution! Daniel (a moderator of announce@) David Darj wrote on Wed, Oct 06, 2010 at 20:14:08 +0200: > I'm happy to announce my release of Subversion 1.6.13 Win32 binaries > and installer > > They are available at my website:http://alagazam.net > and also on SourceForge: http://sourceforge.net/projects/win32svn/ > > > Release notes for the 1.6.x release series may be found at: > > http://subversion.apache.org/docs/release-notes/1.6.html > > You can find the list of changes between 1.6.13 and earlier versions at: > > http://svn.apache.org/repos/asf/subversion/tags/1.6.13/CHANGES > > Questions, comments, and bug reports to us...@subversion.apache.org. > > Regards, > > David Darj > http://alagazam.net > > > >
Re: Subversion 1.6.13 Released
I'm happy to announce my release of Subversion 1.6.13 Win32 binaries and installer They are available at my website:http://alagazam.net and also on SourceForge: http://sourceforge.net/projects/win32svn/ Release notes for the 1.6.x release series may be found at: http://subversion.apache.org/docs/release-notes/1.6.html You can find the list of changes between 1.6.13 and earlier versions at: http://svn.apache.org/repos/asf/subversion/tags/1.6.13/CHANGES Questions, comments, and bug reports to us...@subversion.apache.org. Regards, David Darj http://alagazam.net
Re: Format 20 upgrade to NODES
Greg Stein writes: > What about upgrades from f10 or f19? Upgrading from 1.6 works just as well with NODES as with BASE/WORKING, we write NODES with op_depth 0 or 2. Ugrading from wcng 19 to 20 is very simple, we just copy into NODES with op_depth 0 or 2. -- Philip
Re: svn commit: r1005052 - /subversion/trunk/subversion/tests/libsvn_wc/entries-compat.c
On Wed, Oct 6, 2010 at 10:22, wrote: > Author: julianfoad > Date: Wed Oct 6 14:22:11 2010 > New Revision: 1005052 > > URL: http://svn.apache.org/viewvc?rev=1005052&view=rev > Log: > * subversion/tests/libsvn_wc/entries-compat.c > (TESTING_DATA): Remove erroneous spaces from within a checksum blob. > Make indentation and line breaks more consistent with each other and > with db-test.c, at the expense of one line going over 80 characters. Because you're mixing functional and whitespace changes, this commit is effectively unreviewable using this email. To review it, I would have to go to the cmdline and use the -x -b switch to review the functionality. That isn't going to happen. While this is a minor fix, and doesn't really require careful review, the process of joining function and whitespace changes into a single commit is problematic and should be avoided. >... Thanks, -g
Re: svn commit: r1004792 - /subversion/trunk/notes/wc-ng/nodes
On Wed, Oct 6, 2010 at 03:29, Philip Martin wrote: > Greg Stein writes: > >> On Tue, Oct 5, 2010 at 16:54, Julian Foad wrote: >>>... >>> Earlier today on IRC, Philip and I came to the conclusion that a copy of >>> a mixed-rev subtree (at least from BASE) should be all at the *same* >>> op_depth. >> >> Right. This is why the original NODES table had copyfrom_rev in it -- >> to support copies of mixed-rev subtrees. > > NODES still contains the copyfrom revision, it's the revision column > when op_depth > 0. Aware of that. At one point, it was considered to be "left behind" in the WORKING_NODE table (since it doesn't apply to BASE nodes). But when I started thinking about mixed-rev working copies, I moved the copyfrom_* fields into the NODES definition. Later, when you guys found that a single table solution was best, then it was going in no matter what. I'm just saying that it was already there to support mixed-rev, so I'm simply confirming your conclusion of a single op_depth for copy operations. Cheers, -g
Re: Format 20 upgrade to NODES
What about upgrades from f10 or f19? On Wed, Oct 6, 2010 at 04:32, Philip Martin wrote: > I'd like to enable NODES as a replacement for BASE_NODE and > WORKING_NODE. This would involve bumping the format number, and old > working copies would get automatically upgraded. > > This is not the final NODES data model. It currently just uses > NODES.op_depth as 0 or 2 to indicate the equivalent of BASE_NODE and > WORKING_NODE. Another wc upgrade will be required in the future to > enable full op_depth behaviour. > > The advantages of the proposed upgrade include: dropping the > conditional code, having everyone exercise the new code. > > The disadvantages include: a wc upgrade, the testsuite is slightly > (maybe 2% on my machine) slower. > > If you build the current format 19 with SVN_WC__NODES then both NODES > and BASE_NODE/WORKING_NODE tables are created. DB writes modify both > NODES and BASE/WORKING and DB reads check that the same data is > obtained from NODES and BASE/WORKING. The regression tests pass like > this so we have confidence that NODES is an adequate replacement for > BASE/WORKING. > > If you build format 19 with both SVN_WC__NODES and SVN_WC__NODES_ONLY > then only the NODES table is written and read, BASE/WORKING remain > empty. The upgrade would be using this code, but would also enable > the upgrade and dropping of the BASE/WORKING tables. The change would > be the patch below: > > * subversion/libsvn_wc/wc.h > (SVN_WC__VERSION): Bump to 20. > > * subversion/libsvn_wc/wc.h > (STMT_UPGRADE_TO_20): Renamed from DISABLED_STMT_UPGRADE_TO_20 and > enabled, drop the "inherited" BASE_NODE format 99 stuff. > > > Index: subversion/libsvn_wc/wc.h > === > --- subversion/libsvn_wc/wc.h (revision 1004712) > +++ subversion/libsvn_wc/wc.h (working copy) > @@ -129,7 +129,7 @@ > * Please document any further format changes here. > */ > > -#define SVN_WC__VERSION 19 > +#define SVN_WC__VERSION 20 > > /* Formats <= this have no concept of "revert text-base/props". */ > #define SVN_WC__NO_REVERT_FILES 4 > Index: subversion/libsvn_wc/wc-metadata.sql > === > --- subversion/libsvn_wc/wc-metadata.sql (revision 1004712) > +++ subversion/libsvn_wc/wc-metadata.sql (working copy) > @@ -987,10 +987,8 @@ > > /* Format 20 introduces NODES and removes BASE_NODE and WORKING_NODE */ > > -/* ### Enable this bit and take out the BASE_NODE stuff in format 99 below. > +-- STMT_UPGRADE_TO_20 > > --- DISABLED_STMT_UPGRADE_TO_20 > - > INSERT INTO NODES > SELECT wc_id, local_relpath, 0 AS op_depth, parent_relpath, > repos_id, repos_relpath, revnum, > @@ -1012,7 +1010,6 @@ > DROP TABLE WORKING_NODE; > > PRAGMA user_version = 20; > -*/ > > /* - > */ > > @@ -1028,81 +1025,6 @@ > number will be, however, so we're just marking it as 99 for now. */ > -- format: 99 > > -/* We cannot directly remove columns, so we use a temporary table instead. */ > -/* First create the temporary table without the undesired column(s). */ > -CREATE TEMPORARY TABLE BASE_NODE_BACKUP( > - wc_id INTEGER NOT NULL, > - local_relpath TEXT NOT NULL, > - repos_id INTEGER, > - repos_relpath TEXT, > - parent_relpath TEXT, > - presence TEXT NOT NULL, > - kind TEXT NOT NULL, > - revnum INTEGER, > - checksum TEXT, > - translated_size INTEGER, > - changed_rev INTEGER, > - changed_date INTEGER, > - changed_author TEXT, > - depth TEXT, > - symlink_target TEXT, > - last_mod_time INTEGER, > - properties BLOB, > - dav_cache BLOB, > - file_external TEXT > -); > - > -/* Copy everything into the temporary table. */ > -INSERT INTO BASE_NODE_BACKUP SELECT > - wc_id, local_relpath, repos_id, repos_relpath, parent_relpath, presence, > - kind, revnum, checksum, translated_size, changed_rev, changed_date, > - changed_author, depth, symlink_target, last_mod_time, properties, > dav_cache, > - file_external > -FROM BASE_NODE; > - > -/* Drop the original table. */ > -DROP TABLE BASE_NODE; > - > -/* Recreate the original table, this time less the temporary columns. > - Column descriptions are same as BASE_NODE in format 12 */ > -CREATE TABLE BASE_NODE( > - wc_id INTEGER NOT NULL REFERENCES WCROOT (id), > - local_relpath TEXT NOT NULL, > - repos_id INTEGER REFERENCES REPOSITORY (id), > - repos_relpath TEXT, > - parent_relpath TEXT, > - presence TEXT NOT NULL, > - kind TEXT NOT NULL, > - revnum INTEGER, > - checksum TEXT, > - translated_size INTEGER, > - changed_rev INTEGER, > - changed_date INTEGER, > - changed_author TEXT, > - depth TEXT, > - symlink_target TEXT, > - last_mod_time INTEGER, > - properties BLOB, > - dav_cache BLOB, > - file_external TEXT, > - > - PRIMARY KEY (wc_id, local_relpath) > - ); > - > -/* Recreate the index. */ > -CREATE INDEX I_PARENT
Re: Change the order of NODES table columns
I (Julian Foad) wrote: > I'm planning to change the order of NODES columns so that they're > grouped more logically, like in my BASE_NODE overview comment. If I > can, I'll do this before switching to NODES as the default, as it is > almost impossible to do later. Done in r1005102, r1005104, r1005106, r1005107. - Julian > Currently: > > wc_id 1 > local_relpath 2 > op_depth3 > parent_relpath 4 > repos_id5 > repos_path 6 > revision7 > presence8 > depth 9 -> 13 > moved_here 10 -> (9, > moved_to11 -> 10, > kind12 -> 11) > changed_revision13 -> (16, > changed_date14 -> 17, > changed_author 15 -> 18) > checksum16 -> 14 > properties 17 -> 12 > translated_size 18 -> 19 > last_mod_time 19 -> 20 > dav_cache 20 -> 21 > symlink_target 21 -> 15 > file_external 22 > > Wanted: > > # Indexing > wc_id 1 > local_relpath 2 > op_depth3 > parent_relpath 4 > > # Node-Rev > repos_id5 > repos_path 6 > revision7 > > # Restructuring > presence8 > moved_here 9 <- (10, > moved_to10 <- 11, > > # Content > kind11 <- 12) > properties 12 <- 17 > depth 13 <- 9 > checksum14 <- 16 > symlink_target 15 <- 21 > > # Last-Change > changed_revision16 <- (13, > changed_date17 <- 14, > changed_author 18 <- 15) > > # Misc. cached data > translated_size 19 <- 18 > last_mod_time 20 <- 19 > dav_cache 21 <- 20 > file_external 22 > > Any comments? > > - Julian
Re: svn commit: r1005065 - in /subversion/branches/gpg-agent-password-store: ./ build/generator/ subversion/include/ subversion/include/private/ subversion/libsvn_auth_gpg_agent/ subversion/libsvn_sub
Hi Daniel, Daniel Shahaf wrote: Needs "Patch by:"? Edited the log. I missed it in my log file which I edited in a hurry :( Thank You. -- Senthil Kumaran S http://www.stylesen.org/
RE: Format 20 upgrade to NODES
> -Original Message- > From: Erik Huelsmann [mailto:ehu...@gmail.com] > Sent: woensdag 6 oktober 2010 17:46 > To: Julian Foad > Cc: Philip Martin; dev@subversion.apache.org > Subject: Re: Format 20 upgrade to NODES > > On Wed, Oct 6, 2010 at 1:12 PM, Julian Foad > wrote: > > On Wed, 2010-10-06 at 09:32 +0100, Philip Martin wrote: > >> I'd like to enable NODES as a replacement for BASE_NODE and > >> WORKING_NODE. This would involve bumping the format number, and old > >> working copies would get automatically upgraded. > > > > +1 from me, ASAP. > > > > We're still working on the op_depth support and it's more complex > than I > > originally thought. It looks like doing this transition in two > separate > > format bumps will be more expedient. > > > > Please give me 24h to change the order of NODES columns first - see > > separate email. > > +1 from me too. +1. Keeping the wc_db code operational on two separate database formats is slowing down development and makes it very hard to really start taking advantage of the new format. Bert
Re: Format 20 upgrade to NODES
On Wed, Oct 6, 2010 at 1:12 PM, Julian Foad wrote: > On Wed, 2010-10-06 at 09:32 +0100, Philip Martin wrote: >> I'd like to enable NODES as a replacement for BASE_NODE and >> WORKING_NODE. This would involve bumping the format number, and old >> working copies would get automatically upgraded. > > +1 from me, ASAP. > > We're still working on the op_depth support and it's more complex than I > originally thought. It looks like doing this transition in two separate > format bumps will be more expedient. > > Please give me 24h to change the order of NODES columns first - see > separate email. +1 from me too. Bye, Erik.
Re: svn commit: r1005065 - in /subversion/branches/gpg-agent-password-store: ./ build/generator/ subversion/include/ subversion/include/private/ subversion/libsvn_auth_gpg_agent/ subversion/libsvn_sub
Needs "Patch by:"? style...@apache.org wrote on Wed, Oct 06, 2010 at 14:41:35 -: > Author: stylesen > Date: Wed Oct 6 14:41:35 2010 > New Revision: 1005065 > > URL: http://svn.apache.org/viewvc?rev=1005065&view=rev > Log: > On the 'gpg-agent-password-store' branch: > > Add GPG-AGENT support for Subversion password caching. > > * Makefile.in > Include support for building libsvn_auth_gpg_agent. > > * build.conf > Include support for building libsvn_auth_gpg_agent. > > * configure.ac > Include support for configuring a build to include gpg-agent > support. > > * build/generator/extractor.py > Add gpg-agent auth related functions. > > * subversion/libsvn_subr/auth.c > Add gpg-agent to the list of password store types supported. > > * subversion/libsvn_subr/simple_providers.c > (svn_auth__simple_save_creds_helper): Check for gpg-agent encrypted >password store. > > * subversion/include/private/svn_auth_private.h > Add password type #define for gpg-agent type. > > * subversion/include/svn_auth.h > (svn_auth_gpg_agent_version, > svn_auth_get_gpg_agent_simple_provider): Add new function > prototypes. > > * subversion/libsvn_auth_gpg_agent/version.c > Provide version information for libsvn_auth_gpg_agent. > > * subversion/libsvn_auth_gpg_agent/gpg_agent.c > Password handling functions for gpg-agent password store. > > Added: > > subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/ > > subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/gpg_agent.c > > subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/version.c > Modified: > subversion/branches/gpg-agent-password-store/Makefile.in > subversion/branches/gpg-agent-password-store/build.conf > subversion/branches/gpg-agent-password-store/build/generator/extractor.py > subversion/branches/gpg-agent-password-store/configure.ac > > subversion/branches/gpg-agent-password-store/subversion/include/private/svn_auth_private.h > subversion/branches/gpg-agent-password-store/subversion/include/svn_auth.h > subversion/branches/gpg-agent-password-store/subversion/libsvn_subr/auth.c > > subversion/branches/gpg-agent-password-store/subversion/libsvn_subr/simple_providers.c
Re: svn commit: r1005065 - in /subversion/branches/gpg-agent-password-store: ./ build/generator/ subversion/include/ subversion/include/private/ subversion/libsvn_auth_gpg_agent/ subversion/libsvn_sub
On Wed, Oct 06, 2010 at 02:41:35PM -, style...@apache.org wrote:
> Author: stylesen
> Date: Wed Oct 6 14:41:35 2010
> New Revision: 1005065
>
> URL: http://svn.apache.org/viewvc?rev=1005065&view=rev
> Log:
> On the 'gpg-agent-password-store' branch:
>
> Add GPG-AGENT support for Subversion password caching.
Hi Senthil,
some review inline below.
> Added:
> subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/gpg_agent.c
> URL:
> http://svn.apache.org/viewvc/subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/gpg_agent.c?rev=1005065&view=auto
> ==
> ---
> subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/gpg_agent.c
> (added)
> +++
> subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/gpg_agent.c
> Wed Oct 6 14:41:35 2010
> @@ -0,0 +1,248 @@
> +/*
> + * gpg_agent.c: GPG Agent provider for SVN_AUTH_CRED_*
> + *
> + *
> + *Licensed to the Apache Software Foundation (ASF) under one
> + *or more contributor license agreements. See the NOTICE file
> + *distributed with this work for additional information
> + *regarding copyright ownership. The ASF licenses this file
> + *to you under the Apache License, Version 2.0 (the
> + *"License"); you may not use this file except in compliance
> + *with the License. You may obtain a copy of the License at
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + *Unless required by applicable law or agreed to in writing,
> + *software distributed under the License is distributed on an
> + *"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> + *KIND, either express or implied. See the License for the
> + *specific language governing permissions and limitations
> + *under the License.
> + *
> + */
> +
> +/* */
> +
> +
> +
> +/*** Includes. ***/
> +
> +#include
> +
> +#include
> +#include
> +
> +#include
> +#include "svn_auth.h"
> +#include "svn_config.h"
> +#include "svn_error.h"
> +#include "svn_pools.h"
> +#include "svn_cmdline.h"
> +#include "svn_checksum.h"
> +#include "svn_string.h"
> +
> +#include "private/svn_auth_private.h"
> +
> +#include "svn_private_config.h"
> +
> +static const int buffer_size = 1024;
Please write the constant in upper case: BUFFER_SIZE
And maybe use a #define instead?
> +
> +/* Implementation of password_get_t that retrieves the password
> + from gpg-agent */
> +static svn_boolean_t
> +password_get_gpg_agent(const char **password,
> + apr_hash_t *creds,
> + const char *realmstring,
> + const char *username,
> + apr_hash_t *parameters,
> + svn_boolean_t non_interactive,
> + apr_pool_t *pool)
> +{
> + int sd;
> + char *gpg_agent_info = NULL;
> + char *value;
> + char *p = NULL;
> + char *ep = NULL;
> + char *buffer;
> +
> + apr_array_header_t *socket_details;
> + char *request = NULL;
> + const char *cache_id = NULL;
> + struct sockaddr_un addr;
> + int recvd;
> + char *tty_name;
> + char *tty_type;
> + const char *socket_name = NULL;
> + svn_checksum_t *digest = NULL;
> +
> + value = getenv( "GPG_AGENT_INFO");
> +
> + if (value != NULL)
> +{
> + gpg_agent_info = apr_pstrmemdup(pool, value, strlen(value));
> + socket_details = svn_cstring_split(gpg_agent_info, ":", TRUE, pool);
> + socket_name = APR_ARRAY_IDX(socket_details, 0, const char *);
> +}
> + else
> +return FALSE;
> +
> + value = getenv("GPG_TTY");
> + if (value != NULL)
> +tty_name = apr_pstrmemdup(pool, value, strlen(value));
> + else
> +return FALSE;
> +
> + value = getenv("TERM");
> + if (value != NULL)
> +tty_type = apr_pstrmemdup(pool, value, strlen(value));
> + else
> +return FALSE;
> +
> + if (socket_name != NULL)
> +{
> + addr.sun_family = AF_UNIX;
> + strncpy(addr.sun_path, socket_name, 108);
Using 108 is dangerous, because the buffer isn't guaranteed to be 109 bytes
long. E.g. on OpenBSD it's 104. In this case, we could overrun the buffer.
On Linux it's 108, and because strncpy() does not nul-terminate the
string if the nul doesn't fit, the above risks a non-terminated string.
So please use (sizeof(addr.sun_path) - 1) instead (- 1 to account for '\0').
This seems to be common practice, and aparrently sun_path is always
defined as an array rather than a pointer.
See also http://mail-index.netbsd.org/tech-kern/1997/02/25/0014.html
which explains historical reasons for the different buffer sizes between
systems.
Also, since strncpy() does not guarantee to nul-terminate the buffer,
pleas
Re: svn commit: r1005065 - in /subversion/branches/gpg-agent-password-store: ./ build/generator/ subversion/include/ subversion/include/private/ subversion/libsvn_auth_gpg_agent/ subversion/libsvn_sub
style...@apache.org writes:
> Author: stylesen
> Date: Wed Oct 6 14:41:35 2010
> New Revision: 1005065
> +static svn_boolean_t
> +password_get_gpg_agent(const char **password,
> + apr_hash_t *creds,
> + const char *realmstring,
> + const char *username,
> + apr_hash_t *parameters,
> + svn_boolean_t non_interactive,
> + apr_pool_t *pool)
> +{
> + int sd;
> + char *gpg_agent_info = NULL;
> + char *value;
> + char *p = NULL;
> + char *ep = NULL;
> + char *buffer;
> +
> + apr_array_header_t *socket_details;
> + char *request = NULL;
> + const char *cache_id = NULL;
> + struct sockaddr_un addr;
> + int recvd;
> + char *tty_name;
> + char *tty_type;
> + const char *socket_name = NULL;
> + svn_checksum_t *digest = NULL;
> +
> + value = getenv( "GPG_AGENT_INFO");
> +
> + if (value != NULL)
> +{
> + gpg_agent_info = apr_pstrmemdup(pool, value, strlen(value));
> + socket_details = svn_cstring_split(gpg_agent_info, ":", TRUE, pool);
No need to apr_pstrmemdup if passing to svn_cstring_split.
> + socket_name = APR_ARRAY_IDX(socket_details, 0, const char *);
> +}
> + else
> +return FALSE;
> +
> + value = getenv("GPG_TTY");
> + if (value != NULL)
> +tty_name = apr_pstrmemdup(pool, value, strlen(value));
> + else
> +return FALSE;
> +
> + value = getenv("TERM");
> + if (value != NULL)
> +tty_type = apr_pstrmemdup(pool, value, strlen(value));
> + else
> +return FALSE;
Do you need to apr_pstrmemdup?
> +
> + if (socket_name != NULL)
> +{
> + addr.sun_family = AF_UNIX;
> + strncpy(addr.sun_path, socket_name, 108);
> + sd = socket(AF_UNIX, SOCK_STREAM, 0);
> + if (sd == -1)
> +return FALSE;
> +
> + if (connect(sd, (struct sockaddr *)&addr, sizeof(addr)) == -1)
> +{
> + close(sd);
> + return FALSE;
> +}
> +}
> + else
> +return FALSE;
Use APR's socket interface?
> +
> + /* Receive the connection status from the gpg-agent daemon. */
> + buffer = apr_palloc(pool, buffer_size);
> + recvd = recv(sd, buffer, buffer_size-1, 0);
> + buffer[recvd] = '\0';
> +
> + if (strncmp(buffer, "OK", 2) != 0)
> +return FALSE;
> +
> + /* Send TTY_NAME to the gpg-agent daemon. */
> + request = apr_psprintf(pool, "OPTION ttyname=%s\n", tty_name);
> + send(sd, request, strlen(request), 0);
> + recvd = recv(sd, buffer, buffer_size - 1, 0);
> + buffer[recvd] = '\0';
> +
> + if (strncmp(buffer, "OK", 2) != 0)
> +return FALSE;
> +
> + /* Send TTY_TYPE to the gpg-agent daemon. */
> + request = apr_psprintf(pool, "OPTION ttytype=%s\n", tty_type);
> + send(sd, request, strlen(request), 0);
> + recvd = recv(sd, buffer, buffer_size - 1, 0);
> + buffer[recvd] = '\0';
> +
> + if (strncmp(buffer, "OK", 2) != 0)
> +return FALSE;
> +
> + /* Create the CACHE_ID which will be generated based on REALMSTRING similar
> + to other password caching mechanisms. */
> + digest = svn_checksum_create(svn_checksum_md5, pool);
> + svn_checksum(&digest, svn_checksum_md5, realmstring, strlen(realmstring),
> + pool);
> + cache_id = svn_checksum_to_cstring(digest, pool);
> +
> + if (non_interactive)
> +request = apr_psprintf(pool,
> + "GET_PASSPHRASE --data --no-ask %s X Password:
> \n",
> + cache_id);
> + else
> +request = apr_psprintf(pool,
> + "GET_PASSPHRASE --data %s X Password: \n",
> + cache_id);
> +
> + send(sd, request, strlen(request)+1, 0);
> + recvd = recv(sd, buffer, buffer_size - 1, 0);
> + buffer[recvd] = '\0';
> +
> + if (strncmp(buffer, "ERR", 3) == 0)
> +return FALSE;
> +
> + if (strncmp(buffer, "D", 1) == 0)
> +p = &buffer[2];
> +
> + ep = strchr(p, '\n');
> + if (ep != NULL)
> +*ep = '\0';
> +
> + *password = apr_pstrmemdup(pool, p, recvd);
buffer, and so p, is already allocated from pool.
> +
> + close(sd);
> + return TRUE;
> Added:
> subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/version.c
> URL:
> http://svn.apache.org/viewvc/subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/version.c?rev=1005065&view=auto
> ==
> ---
> subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/version.c
> (added)
> +++
> subversion/branches/gpg-agent-password-store/subversion/libsvn_auth_gpg_agent/version.c
> Wed Oct 6 14:41:35 2010
> @@ -0,0 +1,30 @@
> +/*
> + * version.c: libsvn_auth_gpg_agent version number
> + *
> + *
> + * Copyright (c) 2008 CollabNet. All rights reserved.
> + *
> + * This software is licensed as described in the file COPYING, which
> + * you should ha
Re: [PATCH] gpg-agent support
On Wed, Oct 06, 2010 at 08:02:44PM +0530, Senthil Kumaran S wrote: > I ve reviewed this patch and made many changes to it. For a short > time I would like to commit these changes to a separate branch ie., > https://svn.apache.org/repos/asf/subversion/branches/gpg-agent-password-store > It will be in this branch for a couple of days when I shall do more > tests and then merge it to trunk. > > After looking deep inside this solution, I came to a conclusion that > gpg-agent can give a platform/desktop env independent solution for > password caching. gpg-agent may work in doze too! +1 I'll try to review the branch. Thanks! Stefan
Re: Subversion issue 890 [custom keywords]
On Tue, Oct 05, 2010 at 11:45:18AM -0700, David O'Brien wrote:
> On Wed, Sep 22, 2010 at 08:07:52PM +0200, Stefan Sperling wrote:
> > On Wed, Sep 22, 2010 at 07:52:46AM -0700, David O'Brien wrote:
> > > http://subversion.tigris.org/issues/show_bug.cgi?id=890
> ..
> > However, the patch needs some work. I suppose it has not been synced
> > to Subversion trunk yet, so that will need to be done.
> > Would you be willing to do this?
>
> Hi Stefan,
> Yes, I will look into doing that. As you mention below, the patch is
> still using the 1.6 API and I don't believe anyone has looked at what
> it will take to use the 1.7 API.
>
> Thank you for pointing out various things about the APIs used in the
> patch. That is a big help.
Ok, I'll wait for an updated patch, then. Thank you!
> > > diff -ruN subversion/libsvn_wc/merge.c subversion/libsvn_wc/merge.c
> > > --- subversion/libsvn_wc/merge.c 2009-02-16 23:35:25.0 +0300
> > > +++ subversion/libsvn_wc/merge.c 2009-04-03 22:11:01.30750 +0400
> > > @@ -369,7 +369,7 @@
> > >target_marker,
> > >right_marker,
> > >"===", /* separator */
> > > -
> > > svn_diff_conflict_display_modified_latest,
> > > +
> > > svn_diff_conflict_display_modified_original_latest,
> >
> > The above change looks unrelated to issue #890.
>
> Correct. Should an issue be raised for this?
I don't know if there already is an issue related to custom
conflict markers. If you cannot find one, feel free to file one.
> The API support various conflict markers, but I can find no way to
> configure the svn client on which to use. Thus we have to patch the
> code directly as we've chose "Perforce"-style markers as being most
> useful to our community (both FreeBSD and $WORK).
>
> I wonder if with 1.7 it wouldn't be a good point to change the default?
I'd rather add configuration options than changing the default.
> > > + * Since we're adding freebsd-specific tokens to the log message,
> > > + * clean out any leftovers to avoid accidently sending them to other
> > > + * projects that won't be expecting them.
> > > + */
> > > +
> > > +#define NPREFIX 7
> > > +char *prefixes[NPREFIX] = {
> > > + "PR:",
> ..
> > Your log message template changes are interesting but unrelated to issue
> > #890.
> > The related issue would be:
> > http://subversion.tigris.org/issues/show_bug.cgi?id=1973
>
> Correct -- it is my understanding this issue is already being worked on
> for a future version of Subversion. I fully acknowledge this change is
> not generic and not suitable for inclusion in the stock Subversion.
The request is being tracked, but that doesn't imply that anyone is
working on it. I don't know if anyone is actively working on this.
> I have had to make a similar change at $WORK as many folks depend on the
> template to remind them the various markers we put in our log messages.
>
> So the feature is a highly desired one from the two communities I am
> most active in.
Feel free to work towards a more generic solution, if you have the time
to spare.
I suppose it would help if a design discussion was started on this
mailing list before any actual coding is done. It's important to make
sure the svn community agrees with the approach before doing any serious work.
This is usually low overhead if the design proposal is well thought out.
Usually the design, no matter how good it already is, benefits a lot from
community discussion. Also, you can regard developers involved in the
discussion as potential candidates for reviewing related patches :)
Thanks,
Stefan
Re: Format 20 upgrade to NODES
"Hyrum K. Wright" writes: > wrote: >> >> The disadvantages include: a wc upgrade, the testsuite is slightly >> (maybe 2% on my machine) slower. > > Are there any explanations for this behavior? I haven't profiled it. It could be the NODES queries that do "op_depth = (SELECT MAX(op_depth)" as that's probably more expensive than accessing WORKING_NODE. -- Philip
Re: [PATCH] gpg-agent support
Hi, Dan Engel wrote: [[[ Add GPG-AGENT support for Subversion password handling. * Makefile.in Include support for building libsvn_auth_gpg_agent. * build.conf Include support for building libsvn_auth_gpg_agent. * configure.ac Include support for configuring a build to include gpg-agent support. * subversion/include/private/svn_auth_private.h Add password type #define for gpg-agent type. * subversion/include/svn_auth.h Add declarations of functions provided by libsvn_auth_gpg_agent library. * subversion/libsvn_auth_gpg_agent/gpg_agent.c Password handling functions for gpg-agent password store. * subversion/libsvn_auth_gpg_agent/version.c Provide version information for libsvn_auth_gpg_agent. * subversion/libsvn_subr/auth.c Add gpg-agent to the list of password store types supported. * subversion/libsvn_subr/simple_providers.c Add some special handling for gpg-agent type password. ]]] I ve reviewed this patch and made many changes to it. For a short time I would like to commit these changes to a separate branch ie., https://svn.apache.org/repos/asf/subversion/branches/gpg-agent-password-store It will be in this branch for a couple of days when I shall do more tests and then merge it to trunk. After looking deep inside this solution, I came to a conclusion that gpg-agent can give a platform/desktop env independent solution for password caching. gpg-agent may work in doze too! Thank You. -- Senthil Kumaran S http://www.stylesen.org/
Re: Format 20 upgrade to NODES
On Wed, Oct 6, 2010 at 3:32 AM, Philip Martin wrote: > I'd like to enable NODES as a replacement for BASE_NODE and > WORKING_NODE. This would involve bumping the format number, and old > working copies would get automatically upgraded. > > This is not the final NODES data model. It currently just uses > NODES.op_depth as 0 or 2 to indicate the equivalent of BASE_NODE and > WORKING_NODE. Another wc upgrade will be required in the future to > enable full op_depth behaviour. > > The advantages of the proposed upgrade include: dropping the > conditional code, having everyone exercise the new code. > > The disadvantages include: a wc upgrade, the testsuite is slightly > (maybe 2% on my machine) slower. Are there any explanations for this behavior? > If you build the current format 19 with SVN_WC__NODES then both NODES > and BASE_NODE/WORKING_NODE tables are created. DB writes modify both > NODES and BASE/WORKING and DB reads check that the same data is > obtained from NODES and BASE/WORKING. The regression tests pass like > this so we have confidence that NODES is an adequate replacement for > BASE/WORKING. > > If you build format 19 with both SVN_WC__NODES and SVN_WC__NODES_ONLY > then only the NODES table is written and read, BASE/WORKING remain > empty. The upgrade would be using this code, but would also enable > the upgrade and dropping of the BASE/WORKING tables. The change would > be the patch below: > > * subversion/libsvn_wc/wc.h > (SVN_WC__VERSION): Bump to 20. > > * subversion/libsvn_wc/wc.h > (STMT_UPGRADE_TO_20): Renamed from DISABLED_STMT_UPGRADE_TO_20 and > enabled, drop the "inherited" BASE_NODE format 99 stuff. > > > Index: subversion/libsvn_wc/wc.h > === > --- subversion/libsvn_wc/wc.h (revision 1004712) > +++ subversion/libsvn_wc/wc.h (working copy) > @@ -129,7 +129,7 @@ > * Please document any further format changes here. > */ > > -#define SVN_WC__VERSION 19 > +#define SVN_WC__VERSION 20 > > /* Formats <= this have no concept of "revert text-base/props". */ > #define SVN_WC__NO_REVERT_FILES 4 > Index: subversion/libsvn_wc/wc-metadata.sql > === > --- subversion/libsvn_wc/wc-metadata.sql (revision 1004712) > +++ subversion/libsvn_wc/wc-metadata.sql (working copy) > @@ -987,10 +987,8 @@ > > /* Format 20 introduces NODES and removes BASE_NODE and WORKING_NODE */ > > -/* ### Enable this bit and take out the BASE_NODE stuff in format 99 below. > +-- STMT_UPGRADE_TO_20 > > --- DISABLED_STMT_UPGRADE_TO_20 > - > INSERT INTO NODES > SELECT wc_id, local_relpath, 0 AS op_depth, parent_relpath, > repos_id, repos_relpath, revnum, > @@ -1012,7 +1010,6 @@ > DROP TABLE WORKING_NODE; > > PRAGMA user_version = 20; > -*/ > > /* - > */ > > @@ -1028,81 +1025,6 @@ > number will be, however, so we're just marking it as 99 for now. */ > -- format: 99 > > -/* We cannot directly remove columns, so we use a temporary table instead. */ > -/* First create the temporary table without the undesired column(s). */ > -CREATE TEMPORARY TABLE BASE_NODE_BACKUP( > - wc_id INTEGER NOT NULL, > - local_relpath TEXT NOT NULL, > - repos_id INTEGER, > - repos_relpath TEXT, > - parent_relpath TEXT, > - presence TEXT NOT NULL, > - kind TEXT NOT NULL, > - revnum INTEGER, > - checksum TEXT, > - translated_size INTEGER, > - changed_rev INTEGER, > - changed_date INTEGER, > - changed_author TEXT, > - depth TEXT, > - symlink_target TEXT, > - last_mod_time INTEGER, > - properties BLOB, > - dav_cache BLOB, > - file_external TEXT > -); > - > -/* Copy everything into the temporary table. */ > -INSERT INTO BASE_NODE_BACKUP SELECT > - wc_id, local_relpath, repos_id, repos_relpath, parent_relpath, presence, > - kind, revnum, checksum, translated_size, changed_rev, changed_date, > - changed_author, depth, symlink_target, last_mod_time, properties, > dav_cache, > - file_external > -FROM BASE_NODE; > - > -/* Drop the original table. */ > -DROP TABLE BASE_NODE; > - > -/* Recreate the original table, this time less the temporary columns. > - Column descriptions are same as BASE_NODE in format 12 */ > -CREATE TABLE BASE_NODE( > - wc_id INTEGER NOT NULL REFERENCES WCROOT (id), > - local_relpath TEXT NOT NULL, > - repos_id INTEGER REFERENCES REPOSITORY (id), > - repos_relpath TEXT, > - parent_relpath TEXT, > - presence TEXT NOT NULL, > - kind TEXT NOT NULL, > - revnum INTEGER, > - checksum TEXT, > - translated_size INTEGER, > - changed_rev INTEGER, > - changed_date INTEGER, > - changed_author TEXT, > - depth TEXT, > - symlink_target TEXT, > - last_mod_time INTEGER, > - properties BLOB, > - dav_cache BLOB, > - file_external TEXT, > - > - PRIMARY KEY (wc_id, local_relpath) > - ); > - > -/* Recreate the index. */ > -CREATE INDE
Re: Format 20 upgrade to NODES
On Wed, 2010-10-06 at 09:32 +0100, Philip Martin wrote: > I'd like to enable NODES as a replacement for BASE_NODE and > WORKING_NODE. This would involve bumping the format number, and old > working copies would get automatically upgraded. +1 from me, ASAP. We're still working on the op_depth support and it's more complex than I originally thought. It looks like doing this transition in two separate format bumps will be more expedient. Please give me 24h to change the order of NODES columns first - see separate email. - Julian > This is not the final NODES data model. It currently just uses > NODES.op_depth as 0 or 2 to indicate the equivalent of BASE_NODE and > WORKING_NODE. Another wc upgrade will be required in the future to > enable full op_depth behaviour. > > The advantages of the proposed upgrade include: dropping the > conditional code, having everyone exercise the new code. > > The disadvantages include: a wc upgrade, the testsuite is slightly > (maybe 2% on my machine) slower. > > If you build the current format 19 with SVN_WC__NODES then both NODES > and BASE_NODE/WORKING_NODE tables are created. DB writes modify both > NODES and BASE/WORKING and DB reads check that the same data is > obtained from NODES and BASE/WORKING. The regression tests pass like > this so we have confidence that NODES is an adequate replacement for > BASE/WORKING. > > If you build format 19 with both SVN_WC__NODES and SVN_WC__NODES_ONLY > then only the NODES table is written and read, BASE/WORKING remain > empty. The upgrade would be using this code, but would also enable > the upgrade and dropping of the BASE/WORKING tables. The change would > be the patch below: > > * subversion/libsvn_wc/wc.h > (SVN_WC__VERSION): Bump to 20. > > * subversion/libsvn_wc/wc.h > (STMT_UPGRADE_TO_20): Renamed from DISABLED_STMT_UPGRADE_TO_20 and >enabled, drop the "inherited" BASE_NODE format 99 stuff. [...]
Change the order of NODES table columns
I'm planning to change the order of NODES columns so that they're grouped more logically, like in my BASE_NODE overview comment. If I can, I'll do this before switching to NODES as the default, as it is almost impossible to do later. Currently: wc_id local_relpath op_depth parent_relpath repos_id repos_path revision presence depth moved_here moved_to kind changed_revision changed_date changed_author checksum properties translated_size last_mod_time dav_cache symlink_target file_external Wanted: # Indexing wc_id local_relpath op_depth parent_relpath # Node-Rev repos_id repos_path revision # Restructuring presence moved_here moved_to # Content kind properties depth checksum symlink_target # Last-Change changed_revision changed_date changed_author # Misc. cached data translated_size last_mod_time dav_cache file_external Any comments? - Julian
Fixing FSFS 'Corrupt node-revision' and 'Corrupt representation' errors
We found some corruption in a FSFS repository we were using at work. I have written a script (attached) to fix most but not all of it. WHAT WERE THE SYMPTOMS? --- The version of mod_dav_svn being used was 1.6.9. A user got an error trying to commit one particular file, and also when attempting to check out a fresh WC. I don't have details of these. Then 'svnadmin verify' was run on the repo, and revealed several corrupt revisions, with the following three kinds of error: * svnadmin: Corrupt node-revision '5-12980.0.r12980/5571' svnadmin: Found malformed header in revision file * svnadmin: Corrupt representation '13001 1496 2082 16645 [...]' svnadmin: Malformed representation header * svnadmin: Reading one svndiff window read beyond the end of the representation There were dozens of the first kind, a few of the second kind and one or two of the third kind. The corrupt revisions were spread over a period of a few weeks, with no corrupt revisions before that or after that. We know of nothing special about that time period. ANALYSIS I used both plain text searching and John Szakmeister's 'fsfsverify.py' to help analyze the revision files. Here are just the brief results of what I found. Most of the 'Corrupt node-revision' errors were due to the byte-offset part of the node-rev id being wrong. This error occurred with many different node-rev ids. A corrupt revision contained from one to several ids with wrong byte-offsets. Each particular node-rev id appeared in several different revisions after the one in which it was created, and it appeared correctly in some of them and wrongly in others, with no discernable pattern. Every time it appeared wrongly, it had the same wrong value, so there were only two variants of each node-rev id: the right one and the wrong one. The byte-offset was always fairly close to the correct value, but off by about 5 to 500 bytes. The wrong byte-offset did not point to any special place in the target revision file, such as the start or end of a data blob, so svnadmin reported 'Found malformed header'. One or two 'Corrupt node-revision' errors were wrong in another way. A directory entry reference to a subdirectory named 'X' (not its real name) had the exact value 'dir 6-12953.0.r12953/30623'. Exactly one of the node-revs created in r12953 was named 'X', and it was a directory at the right path, and its node-rev id was '0-12953.0.r12953/30403'. Therefore I concluded that that is the correct replacement. Note that both the node-id component and the byte-offset part were wrong. The 'Corrupt representation' errors were also due to a byte-offset being wrong. The second number, '1496' in the above example, is supposed to be the byte-offset in the revision file. Like the node-rev byte offsets, these were typically off by a small amount. I did not investigate or fix the 'Reading one svndiff window ...' error. THE SCRIPT TO FIX THE ERRORS Usage: ./fix-repo REPO-DIR START-REVNUM Files (attached, separately and as .tgz): fix-repo# shell script, iterates over rev numbers; calls ... fixer/fix-rev.py# finds and fixes errors, using ... fixer/find_good_id.py # looks up a node-rev id, ignoring offset fixer/__init__.py # empty file, defines this as a Python module When the script sees a 'Corrupt node-revision' error message, it looks up the node-rev id ignoring its offset part. If found, it substitutes the correct full id wherever it occurs in the revision file. It expects this change to result in a checksum error being reported next, and so it substitutes the calculated checksum as reported in the error message. (In fact, it assumes that any checksum error being reported should be simply corrected in this way.) For the second type of 'Corrupt node-revision' error, I could not find a simple rule to determine when a node-rev id was wrong in this way so I hard-coded that one specific substitution into the script. When the script sees a 'Corrupt representation' error, it searches for all representations in the target revision and, if exactly one of them has the expected length, it substitutes the offset of this one. LIMITATIONS & IMPROVEMENTS -- The script's algorithm is crude and could do with improvement in several respects if it is to be used more widely. It doesn't respect checksums. When fixing a node-rev id, it should update only the corresponding checksum rather than assuming that any reported checksum error is the sole result of this fix. When fixing a representation offset, it should ensure the rep that it finds is in fact the right one, probably by checking the checksum. Detecting and fixing the second type of 'Corrupt node-revision' error could probably be automated. It doesn't replace a wrong byte-offset if the correct byte-offset has a different number of digits. I didn't encounter a need for this. This would be ve
Format 20 upgrade to NODES
I'd like to enable NODES as a replacement for BASE_NODE and WORKING_NODE. This would involve bumping the format number, and old working copies would get automatically upgraded. This is not the final NODES data model. It currently just uses NODES.op_depth as 0 or 2 to indicate the equivalent of BASE_NODE and WORKING_NODE. Another wc upgrade will be required in the future to enable full op_depth behaviour. The advantages of the proposed upgrade include: dropping the conditional code, having everyone exercise the new code. The disadvantages include: a wc upgrade, the testsuite is slightly (maybe 2% on my machine) slower. If you build the current format 19 with SVN_WC__NODES then both NODES and BASE_NODE/WORKING_NODE tables are created. DB writes modify both NODES and BASE/WORKING and DB reads check that the same data is obtained from NODES and BASE/WORKING. The regression tests pass like this so we have confidence that NODES is an adequate replacement for BASE/WORKING. If you build format 19 with both SVN_WC__NODES and SVN_WC__NODES_ONLY then only the NODES table is written and read, BASE/WORKING remain empty. The upgrade would be using this code, but would also enable the upgrade and dropping of the BASE/WORKING tables. The change would be the patch below: * subversion/libsvn_wc/wc.h (SVN_WC__VERSION): Bump to 20. * subversion/libsvn_wc/wc.h (STMT_UPGRADE_TO_20): Renamed from DISABLED_STMT_UPGRADE_TO_20 and enabled, drop the "inherited" BASE_NODE format 99 stuff. Index: subversion/libsvn_wc/wc.h === --- subversion/libsvn_wc/wc.h (revision 1004712) +++ subversion/libsvn_wc/wc.h (working copy) @@ -129,7 +129,7 @@ * Please document any further format changes here. */ -#define SVN_WC__VERSION 19 +#define SVN_WC__VERSION 20 /* Formats <= this have no concept of "revert text-base/props". */ #define SVN_WC__NO_REVERT_FILES 4 Index: subversion/libsvn_wc/wc-metadata.sql === --- subversion/libsvn_wc/wc-metadata.sql(revision 1004712) +++ subversion/libsvn_wc/wc-metadata.sql(working copy) @@ -987,10 +987,8 @@ /* Format 20 introduces NODES and removes BASE_NODE and WORKING_NODE */ -/* ### Enable this bit and take out the BASE_NODE stuff in format 99 below. +-- STMT_UPGRADE_TO_20 --- DISABLED_STMT_UPGRADE_TO_20 - INSERT INTO NODES SELECT wc_id, local_relpath, 0 AS op_depth, parent_relpath, repos_id, repos_relpath, revnum, @@ -1012,7 +1010,6 @@ DROP TABLE WORKING_NODE; PRAGMA user_version = 20; -*/ /* - */ @@ -1028,81 +1025,6 @@ number will be, however, so we're just marking it as 99 for now. */ -- format: 99 -/* We cannot directly remove columns, so we use a temporary table instead. */ -/* First create the temporary table without the undesired column(s). */ -CREATE TEMPORARY TABLE BASE_NODE_BACKUP( - wc_id INTEGER NOT NULL, - local_relpath TEXT NOT NULL, - repos_id INTEGER, - repos_relpath TEXT, - parent_relpath TEXT, - presence TEXT NOT NULL, - kind TEXT NOT NULL, - revnum INTEGER, - checksum TEXT, - translated_size INTEGER, - changed_rev INTEGER, - changed_date INTEGER, - changed_author TEXT, - depth TEXT, - symlink_target TEXT, - last_mod_time INTEGER, - properties BLOB, - dav_cache BLOB, - file_external TEXT -); - -/* Copy everything into the temporary table. */ -INSERT INTO BASE_NODE_BACKUP SELECT - wc_id, local_relpath, repos_id, repos_relpath, parent_relpath, presence, - kind, revnum, checksum, translated_size, changed_rev, changed_date, - changed_author, depth, symlink_target, last_mod_time, properties, dav_cache, - file_external -FROM BASE_NODE; - -/* Drop the original table. */ -DROP TABLE BASE_NODE; - -/* Recreate the original table, this time less the temporary columns. - Column descriptions are same as BASE_NODE in format 12 */ -CREATE TABLE BASE_NODE( - wc_id INTEGER NOT NULL REFERENCES WCROOT (id), - local_relpath TEXT NOT NULL, - repos_id INTEGER REFERENCES REPOSITORY (id), - repos_relpath TEXT, - parent_relpath TEXT, - presence TEXT NOT NULL, - kind TEXT NOT NULL, - revnum INTEGER, - checksum TEXT, - translated_size INTEGER, - changed_rev INTEGER, - changed_date INTEGER, - changed_author TEXT, - depth TEXT, - symlink_target TEXT, - last_mod_time INTEGER, - properties BLOB, - dav_cache BLOB, - file_external TEXT, - - PRIMARY KEY (wc_id, local_relpath) - ); - -/* Recreate the index. */ -CREATE INDEX I_PARENT ON BASE_NODE (wc_id, parent_relpath); - -/* Copy everything back into the original table. */ -INSERT INTO BASE_NODE SELECT - wc_id, local_relpath, repos_id, repos_relpath, parent_relpath, presence, - kind, revnum, checksum, translated_size, changed_rev, changed_date, - changed_author, depth, symlink_target, last_mod_time, properties,
Re: svn commit: r1004792 - /subversion/trunk/notes/wc-ng/nodes
Greg Stein writes: > On Tue, Oct 5, 2010 at 16:54, Julian Foad wrote: >>... >> Earlier today on IRC, Philip and I came to the conclusion that a copy of >> a mixed-rev subtree (at least from BASE) should be all at the *same* >> op_depth. > > Right. This is why the original NODES table had copyfrom_rev in it -- > to support copies of mixed-rev subtrees. NODES still contains the copyfrom revision, it's the revision column when op_depth > 0. -- Philip
