Re: SVN: Out of memory when committing
Still happens with 1.6.16. Am I the only one? Any idea, at least, of what is causing this behaviour? On Wed, Feb 23, 2011 at 10:07, Ole Pinto olepi...@gmail.com wrote: Hello, I've just began in a new job and, due to the repo organization, I've found a problem I didn't have before. I'm using the command line version of svn: svn, version 1.6.15 (SlikSvn/1.6.15) WIN32 compiled Dec 17 2010, 14:03:07 I am getting the Out of memory - terminating application message when checking in, without any regard to size of working copy or changes. I have got to the point of knowing what cause it; the repo goes kind of: / ├──j │ ├──1 │ │ ├──release │ │ └──trunk │ ├──2 │ └──3 └──l ├──l1 │ ├──release │ ├──trunk │ ├──v5 │ └──v6 ├──l2 └──l3 I have a local copy of some of it, with the same structure (beginning at root), for example: C:\repo ├──j │ └──1 │ └──trunk └──l └──l1 └──trunk Now I want to make big changes to l/l1, so I cd /repo/l/l1; svn co .../repo/l/l1/trunk my_trunk, change whatever, cd into my_trunk, and the checkin will fail as described. Workarounds: 1) put aside trunk, rename my_trunk to trunk and check in 2) move my_trunk out of the local repository (ie. into /tmp/my_trunk) and commit that. Related, AFAICS, to issue 3540http://subversion.tigris.org/issues/show_bug.cgi?id=3540. I'd say it's a P2 more than a P3, as I have found it every time I have committed. Any easier way? Changing the repo is not an option, but some change in how i keep the working copy could be possible... Regards, Óscar
RE: Unexpected behaviour with SVNPath/SVNParentPath mixture
Hi, -Original Message- From: Cooke, Mark [mailto:mark.co...@siemens.com] Sent: Friday, March 25, 2011 12:38 PM To: Jans Ullrich; subversion-20...@ryandesign.com Cc: users@subversion.apache.org Subject: RE: Unexpected behaviour with SVNPath/SVNParentPath mixture The question is, why not? According to the Apache docs, it should work, so it looks like a problem in subversion. Can this be considered a bug? Should I file an issue? ...because when you use the SVNParentPath directive, apache has no further involvement in any child paths, all paths that start with the parent root are passed to the DAV handler... You can get round this by specifiying separate handlers for the separate paths using multiple SVNPath blocks but again, anything below those paths is handled by DAV and not apache. This is different from nesting access restrictions to artifacts that are all handled by apache (i.e. uri Locations or local Directorys so those docs you mention do not apply in this scenario. Thanks for the explanation. I think I get it now, just the effect with the path component suddenly doubling still puzzles me a bit. I still suspect a bug or misfeature there, since I'd have thought it should either work not at all or work as expected. But since it looks like we have a workaround, I guess I can live with that. We'd like to provide our users with the ability to create repositories themselves, then possibly promote select repositories to a different permission set. Restricting ourselves to only using SVNPath would be inconvenient... ;-) You could consider using one (set of) parent path(s) for restricted repos and another (set) for less restricted ones? I'll have to check that, but I suspect it will be hard because a lot of our build architecture is already using these paths. The modified permissions are the new part. But I think the idea is good, maybe we can have a completely different location for the SVNPath configs (like /svn/parentpathtest_ext/...) then we should have no conflict, should we? Thanks for the idea and the explanation above! Cheers, Ulli Please note: This e-mail may contain confidential information intended solely for the addressee. If you have received this e-mail in error, please do not disclose it to anyone, notify the sender promptly, and delete the message from your system. Thank you.
Re: SVN EQ Recovery
Stefan Sperling wrote on Thu, Mar 24, 2011 at 12:17:19 +0100: For the missing revisions, you will need to manually replay the *exact* changes made in them (using checkout and commit from a working copy), so that future revisions fit on top. If that doesn't get you anywhere, a more low-level approach might work: I'm not sure I agree with this suggestion. (certainly not at this point in time)
Re: SVN EQ Recovery
Other points: * The terms used are unclear. (Example: 'incremental backup' could mean 'zfs/lvm snapshot', could mean find -mtime | xargs tar, and could mean 'incremental dump'.) I'd rather get a concrete set of facts (read: one that doesn't require me to guess what the facts are) before suggesting a recovery procedure. * There are some more subtleties here. One example: under certain conditions (which depend on your configuration and on your data), if you don't enable the rep-cache while re-creating revisions, the pieced-together repository may be corrupted. Stefan Sperling wrote on Thu, Mar 24, 2011 at 12:17:19 +0100: On Tue, Mar 22, 2011 at 08:27:39PM -0700, bdu12 wrote: Hello, I use to have two SVN repositories and a single trac DB setup running in Ubuntu on vmware. The server had a cron daily job that ran each night doing incremental backups onto an email server (the incremental backups, backs up files that have been changed during the day). Occasionally I also did a snapshot of the system for a complete backup. I guess this means that you backed up the repository files as they appeared on the filesystem (i.e. repos/db folders etc.). Which repository backend were you using? BDB or FSFS? See the file called repos/db/fs-type. FSFS has been the default for quite some time so it's likely that you have FSFS repositories. Do you have any dump files of repository data (obtainable via svnadmin dump)? My server has now been lost after the Christchurch earthquake in NZ on 22/02/2011 and am trying to rebuild everything from these backups. I have managed to recover an old snapshot from: - 24/01/2011 I have also managed to recover all incremental backups except for two: - 28/01/2011 - 29/01/2011 I managed to take the client laptops out of the office when the quake hit which have different revisions of the DB's from different checkouts done of the different projects. DB as in Subversion repository? Note that a checkout may not mirror any given revision in the repository. It working copy can contain mixed revisions. See http://svnbook.red-bean.com/nightly/en/svn.basic.in-action.html#svn.basic.in-action.mixedrevs Is there a way of rebuilding my server with all of this data? I am not sure what I was working on on the days that I lost the incremental backups. Once a revision file has been created, it is never changed. Newer revisions often refer to data from older revisions. In FSFS repository filesystems, Subversion stores revision data as separate files in repos/db/revs and repos/db/revprops. So incremental backups of FSFS repositories should only be adding new files to the repository, with one exception. The file repos/db/current contains the number of the HEAD revision, and should change in every incremental backup. If the data you already have tells you enough to figure out what the missing changes were, you can try restoring revisions in order. Use the full backup you have as a starting point. Then copy in new revision files from incremental backups, and also copy or adjust the 'current' file. For the missing revisions, you will need to manually replay the *exact* changes made in them (using checkout and commit from a working copy), so that future revisions fit on top. If that doesn't get you anywhere, a more low-level approach might work: Try to figure out which files and directories were changed in the missing revisions. E.g. try to open every file in every revision (from HEAD downwards) and note the path/revision pairs for which svn errors out. Now locate revisions that changed the same paths. Decode the corresponding revision files from before and after the missing revisions and try to interpolate the changes that the missing revisions were describing. A tool that can decode FSFS revision files is here: https://svn.apache.org/repos/asf/subversion/trunk/contrib/server-side/fsfsverify.py This tool parses FSFS revision files and prints their content in somewhat human-readable form. To understand the decoded data, refer to these documents: http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_base/notes/fs-history http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_base/notes/structure http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure http://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff A similar approach might work for BDB backends, but off-hand I don't know how it would be done. Some of the above links talk about BDB, and information within them is also important for understanding FSFS. I think in theory I should be able to have the current working copy per project and then perform a reverse diff of the revisions from 30/01/2011 to the date just before the checkout of the project in question. This should then give the version at 30/01/2011. Then using this version with the
Re: cvs log equivalent
Johan Corveleyn wrote on Sat, Mar 19, 2011 at 14:50:30 +0100: On Sat, Mar 19, 2011 at 2:21 AM, Hyrum K. Wright hyrum_wri...@mail.utexas.edu wrote: On Fri, Mar 18, 2011 at 6:13 PM, Paul Graham pgra...@oasys-ds.com wrote: I could find all the change versions of a file, then do an svn diff for each change, then parse the output and determine the number of changes, but that seems excessive :-) rcs has this lines+/- information directly in the database. Is svn organized differently under the hood? Significantly. One thing I usually do is run svn diff on the revision of interest and then pipe that through diffstat: $ svn diff -c1089374 | diffstat That will usually yield sufficiently interesting result for my application. Paul, It may interest you to know that the upcoming 1.7 release of svn will have a '--diff' option for 'svn log', which will show the diffs inline with the log output. You may be able to script something around that, post-processing the output of 'svn log --diff' by sending each diff to 'diffstat', and replacing the diff output with the diffstat output or something (this will be much more efficient than executing 'svn diff -c XXX | diffstat' for every revision that's being output by log). I think you mean: svn log --diff --diff-cmd=/path/to/wrapper-script-around-diffstat
Naming scheme for property names/prefixes
Hallo List, I wonder if there is already a naming scheme for property names. Certainly we all know that the subversion native properties have a prefix of svn:. Prefixes of tsvn: for tortoise properties (?) and bugtraq: for bug tracking purposes are also invented. But what about sire internal stuff and systems under development? x-something:? Just site shortname:? Any conventions yet? Best regards Henning Hucke -- I will follow the good side right to the fire, but not into it if I can help it. -- Michel Eyquem de Montaigne
Re: Non-file based checkout?
You probably want to use the svn_repos or svn_ra API, not the FS API directly. That said: see svn_test__set_file_contents(), used by fs-test.c . Clifford Yapp wrote on Thu, Mar 24, 2011 at 13:02:03 -0400: On Thu, Mar 24, 2011 at 12:10 PM, Stefan Sperling s...@elego.de wrote: The LGPLv2 and APLv2 are compatible. Just list both licences in your copyright notices, and you can redistribute the derived work containing both Subversion's code and your own LGPLv2 code. Files from Subversion which you modify must keep the APLv2 licence -- you cannot simply remove a licence, nor change it. But you can clearly mark your own changes and licence them under the LGPLv2 if you think that's worth it. (Note that this would prevent your changes from being merged into Subversion itself.) OK, sounds good - thanks! On a more technical note - I'm looking at svn_fs_apply_text as a way to take binary data and stuff it into a file in the repository created with svn_fs_make_file - is this the correct approach? I basically have a small chunk of binary data I want to stuff into the file and commit, but it looks like I need to turn it into a svn_string_t and then stream that string via a svn_stream_t? Thanks much, CY
Re: Non-file based checkout?
Stefan Sperling wrote on Thu, Mar 24, 2011 at 12:38:10 +0100: On Wed, Mar 23, 2011 at 08:12:57PM -0400, Clifford Yapp wrote: Hi! I'm looking at using the subversion libraries to handle data, and I would like to integrate them into an application for revisioning/managing computer aided design geometry information. Looking over the client API, it seems to assume that a checkout will result in a file in a filesystem path (e.g. a working checkout). Is there a way to request in C the straight-up data from the svn repository without bothering the disk, e.g. checkout svn checkout file://repository/datafile1 and have it returned as a char * array or some other binary form in C? Based on my current understanding I would have to checkout the file in question to a working directory and read it in from there to get it in memory - is that true or is there another, lower level API that doesn't require that step? I have a situation where I have potentially thousands of files, some large and some small, that all need to be individually tracked and then combined into single larger files. It can really kill performance to have to involve the filesystem and write out lots of individual tiny files - ideally I'd like to send information to/from the repository using what would in effect be an in-geometry-storage-file checkout. Any help appreciated - thanks! Take a look at the svn_client_cat2() function. The command-line equivalent is the svn cat command. The resulting file is read-only, so this is suitable for display purposes mosly. You cannot commit changes back to the file, though. For committing with the client API, you really need working copy. But a working copy of the parent directory is sufficient. You could look into using svn_depth_t (the API-equivalent of the --depth command-line option) to create working copies that do not contain data you don't need. Another approach would be going beneath the client API (Subversion is a stack of layers, with a public API at each layer -- so you're not confined to the client API). Take a look at the repository access API in svn_ra.h. Using that API, it should be possible to write a client that does everything in memory. This approach gives you more flexibility. But you will need to reimplement parts of what the client library implements (a commit editor, for example). To get off the ground quicker you could re-use and adjust existing client code (if you're happy with the Apache licence). svnmucc does some of this...
Subversion Performance Benchmarks
Hi, I am working on a tool to establish performance benchmarks for Subversion. Primary goal would be to have some controlled tests for doing release to release comparison. The immediate goal is to come up with something to compare the performance of current release with the development going on in 1.7 but these could be useful for doing other types of comparisons. I have posted a blog entry with details. Take a look, try the tests and provide some feedback. http://blogs.collab.net/subversion/2011/03/subversion-1-7-performance-benchmarks/ Please take a look, try the tests and provide some feedback. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: cvs log equivalent
On Fri, Mar 25, 2011 at 7:34 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Sat, Mar 19, 2011 at 14:50:30 +0100: On Sat, Mar 19, 2011 at 2:21 AM, Hyrum K. Wright hyrum_wri...@mail.utexas.edu wrote: On Fri, Mar 18, 2011 at 6:13 PM, Paul Graham pgra...@oasys-ds.com wrote: I could find all the change versions of a file, then do an svn diff for each change, then parse the output and determine the number of changes, but that seems excessive :-) rcs has this lines+/- information directly in the database. Is svn organized differently under the hood? Significantly. One thing I usually do is run svn diff on the revision of interest and then pipe that through diffstat: $ svn diff -c1089374 | diffstat That will usually yield sufficiently interesting result for my application. Paul, It may interest you to know that the upcoming 1.7 release of svn will have a '--diff' option for 'svn log', which will show the diffs inline with the log output. You may be able to script something around that, post-processing the output of 'svn log --diff' by sending each diff to 'diffstat', and replacing the diff output with the diffstat output or something (this will be much more efficient than executing 'svn diff -c XXX | diffstat' for every revision that's being output by log). I think you mean: svn log --diff --diff-cmd=/path/to/wrapper-script-around-diffstat Ohhh, I hadn't thought of that. That certainly seems like a nice way to do this ... Cheers, -- Johan
Re: cvs log equivalent
The Lines field of svn log is puzzling. It says how many lines of comment, etc., follow the header line. It is therefore about as informative as this page intentionally left blank. Yet in its location in the log message it resembles the cvs lines-changed field enough to be misleading. Paul - Original Message - From: Johan Corveleyn jcor...@gmail.com To: Daniel Shahaf d...@daniel.shahaf.name Cc: Paul Graham pgra...@oasys-ds.com, users@subversion.apache.org Sent: Friday, March 25, 2011 3:24:22 PM Subject: Re: cvs log equivalent On Fri, Mar 25, 2011 at 7:34 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Sat, Mar 19, 2011 at 14:50:30 +0100: On Sat, Mar 19, 2011 at 2:21 AM, Hyrum K. Wright hyrum_wri...@mail.utexas.edu wrote: On Fri, Mar 18, 2011 at 6:13 PM, Paul Graham pgra...@oasys-ds.com wrote: I could find all the change versions of a file, then do an svn diff for each change, then parse the output and determine the number of changes, but that seems excessive :-) rcs has this lines+/- information directly in the database. Is svn organized differently under the hood? Significantly. One thing I usually do is run svn diff on the revision of interest and then pipe that through diffstat: $ svn diff -c1089374 | diffstat That will usually yield sufficiently interesting result for my application. Paul, It may interest you to know that the upcoming 1.7 release of svn will have a '--diff' option for 'svn log', which will show the diffs inline with the log output. You may be able to script something around that, post-processing the output of 'svn log --diff' by sending each diff to 'diffstat', and replacing the diff output with the diffstat output or something (this will be much more efficient than executing 'svn diff -c XXX | diffstat' for every revision that's being output by log). I think you mean: svn log --diff --diff-cmd=/path/to/wrapper-script-around-diffstat Ohhh, I hadn't thought of that. That certainly seems like a nice way to do this ... Cheers, -- Johan
Re: cvs log equivalent
On Fri, Mar 25, 2011 at 3:35 PM, Paul Graham pgra...@oasys-ds.com wrote: The Lines field of svn log is puzzling. It says how many lines of comment, etc., follow the header line. It is therefore about as informative as this page intentionally left blank. Yet in its location in the log message it resembles the cvs lines-changed field enough to be misleading. FWIW, I believe it exists for scripts that might be parsing the output. It lets them know how many lines are in the comment. I agree on the confusion, especially for CVS users. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: Non-file based checkout?
Are those (svn_repos/svn_ra) the API levels that allow safe read/write access from multiple clients to the repository? I did manage to get a basic svn_fs test working (very cool!), but it did look like that might be too low-level for safety. Cheers, CY On Fri, Mar 25, 2011 at 2:48 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: You probably want to use the svn_repos or svn_ra API, not the FS API directly. That said: see svn_test__set_file_contents(), used by fs-test.c .
