Re: Dealing with very old repo format (version 1)
On Wed, Apr 29, 2015 at 1:32 AM, Branko Čibej br...@wandisco.com wrote: On 29.04.2015 07:14, Nico Kadel-Garcia wrote: On Wed, Apr 29, 2015 at 12:03 AM, Branko Čibej br...@wandisco.com wrote: On 29.04.2015 05:09, Nico Kadel-Garcia wrote: On Tue, Apr 28, 2015 at 5:03 PM, Andrew Reedick jreed...@incomm.com wrote: Does anyone have any tips on how to upgrade a very old repo? The db/format lists 1. A 1.8 svn client cannot hotcopy, dump or svnadmin upgrade such an old repo, all of which fail with svnadmin: E720002: Can't open file 'devel\db\current': The system cannot find the file specified. Do I need find a really old svn client (1.3?) and upgrade? Do I need to manually create the db/current file? Supposedly , a format of 1 is from pre-svn 1.0. https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_repos/repos.h - Formats 0, 1 and 2 were pre-1.0. Why can't you do a fresh working copy, and copy all files except those in '.svn' subdirectories? Without db/current, he can't perform a checkout, and if he could and just copied the file, he'd be throwing away all history. What? hotcopy, dump, and svnadmin upgrade are all operations on the repository itself. The database *of the repository* is irrelevant to fresh checkouts, as long as they can speak any relevant network protocols associated with the old service. You proposed copying all files except those in .svn directories from a fresh checkout. This implies actually doing a checkout, which you can't do if your repository is corrupt, and it's corrupt if it's a FSFS repo without a db/current file. It's irrelevant whether you're doing a checkout from a local repository or over the network or via git-svn or whatever. I was assuming his core problem was the age of the repository and finding compatible server side tools to help with the upgrade. I also admit, myself, that when migrating old repositories, I could generally not care less about the history, which has often become quite cluttered with old mistakes and accidental binary commits. If it's genuinely corrupt, that's a distinct issue from all my current tools are unable to read such an old repository, which is what it sounded like. I'll be curious to see if Philip Martin's suggestion to use 'svnadmin recover' helps out with that. I'd also make *absolutely sure* to use rsync -a or cp -a on Linux systems to copy the repository somewhere else, and only touch the *copy*, until this is straightened out.
Re: Dealing with very old repo format (version 1)
On Wed, Apr 29, 2015 at 12:03 AM, Branko Čibej br...@wandisco.com wrote: On 29.04.2015 05:09, Nico Kadel-Garcia wrote: On Tue, Apr 28, 2015 at 5:03 PM, Andrew Reedick jreed...@incomm.com wrote: Does anyone have any tips on how to upgrade a very old repo? The db/format lists 1. A 1.8 svn client cannot hotcopy, dump or svnadmin upgrade such an old repo, all of which fail with svnadmin: E720002: Can't open file 'devel\db\current': The system cannot find the file specified. Do I need find a really old svn client (1.3?) and upgrade? Do I need to manually create the db/current file? Supposedly , a format of 1 is from pre-svn 1.0. https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_repos/repos.h - Formats 0, 1 and 2 were pre-1.0. Why can't you do a fresh working copy, and copy all files except those in '.svn' subdirectories? Without db/current, he can't perform a checkout, and if he could and just copied the file, he'd be throwing away all history. What? hotcopy, dump, and svnadmin upgrade are all operations on the repository itself. The database *of the repository* is irrelevant to fresh checkouts, as long as they can speak any relevant network protocols associated with the old service. And heck, if you need to pull logic this way, use 'git-svn' to pull it into a git repository and push it back to a new, clean, Subversion repository. Unless, Andrew? Do you not have any network based service associated with the old repository?
Re: Dealing with very old repo format (version 1)
On 29.04.2015 07:14, Nico Kadel-Garcia wrote: On Wed, Apr 29, 2015 at 12:03 AM, Branko Čibej br...@wandisco.com wrote: On 29.04.2015 05:09, Nico Kadel-Garcia wrote: On Tue, Apr 28, 2015 at 5:03 PM, Andrew Reedick jreed...@incomm.com wrote: Does anyone have any tips on how to upgrade a very old repo? The db/format lists 1. A 1.8 svn client cannot hotcopy, dump or svnadmin upgrade such an old repo, all of which fail with svnadmin: E720002: Can't open file 'devel\db\current': The system cannot find the file specified. Do I need find a really old svn client (1.3?) and upgrade? Do I need to manually create the db/current file? Supposedly , a format of 1 is from pre-svn 1.0. https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_repos/repos.h - Formats 0, 1 and 2 were pre-1.0. Why can't you do a fresh working copy, and copy all files except those in '.svn' subdirectories? Without db/current, he can't perform a checkout, and if he could and just copied the file, he'd be throwing away all history. What? hotcopy, dump, and svnadmin upgrade are all operations on the repository itself. The database *of the repository* is irrelevant to fresh checkouts, as long as they can speak any relevant network protocols associated with the old service. You proposed copying all files except those in .svn directories from a fresh checkout. This implies actually doing a checkout, which you can't do if your repository is corrupt, and it's corrupt if it's a FSFS repo without a db/current file. It's irrelevant whether you're doing a checkout from a local repository or over the network or via git-svn or whatever. -- Brane
Re: Dealing with very old repo format (version 1)
Branko Čibej br...@wandisco.com writes: In any case, 1.8 /should/ be able to dump an FSFSv1 repository, and the repos/db/current file should exists; it's been around since FSFSv1. You can try recreating it; the format is described here: https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure Running 'svnadmin recover' should recreate the missing file. -- Philip Martin | Subversion Committer WANdisco // *Non-Stop Data*
Re: Cannot checkout or clean up using 1.9 dev build
I’m not sure if the \\?\ format is supported in the new rename api(as this is an option in certain publi apis and not a low level supported format), but it should certainly be used in the fallback codepath where the normal rename is tried, like hiw it is in apr. But if the new api fails with the limit, it is quite likely that it supports the \\?\ format. Some more recent new in Windows 8+ apis just dropped the path length limit and therefore don’t support the \\?\ format… but that doesn’t apply here as this api was new in Vista. I hope somebody looks at this earlier, but I’ll will look into this early next week. Bert Sent from Windows Mail From: Ivan Zhakov Sent: Tuesday, April 28, 2015 9:07 AM To: Benjamin Fritz, Bert Huijben Cc: 'subversion' On 27 April 2015 at 21:35, Benjamin Fritz fritzophre...@gmail.com wrote: Apparently I'm not subscribed to get list emails; please CC me on future responses. I had to get this message from the archive :-) Branko Čibej wrote: On 27.04.2015 18:06, Benjamin Fritz wrote: I get the following error when I try to check out a working copy from my repository at work (exact path names obfuscated but length unchanged, in case it matters It probably does matter ... FWIW, I believe you did change the length of the file name, the workqueue record says 234 chars but the name in the record is 240; still, that shouldn't be an issue. Oops. You're right. My actual directory has only 3 characters where I replaced with Prog in the path string. Still, those 6 fewer characters are probably irrelevant. ), leaving a partially downloaded working copy: svn: E155009: Failed to run the WC DB work queue associated with 'C:\Project_Files\Proj_Dev\Prog_svn_1.9_cmdline', work item 2160 (file-install 234 Logical_Architecture/APP/App_PL_Verification/XXX/XXX_XXXs/XXX_7_APPDB_PROJ_PROG--0021_BE/APPDB_PROJ_PROG--0021.SUP/APPDB_PROJ_PROG--0021/APPDB_PROJ_PROG--0021/APPDB_PROJ_PROG--0021.SUP/XXX_APPDB_PROJ_PROG--002101.CRC 1 0 1 1) svn: E720123: The filename, directory name, or volume label syntax is incorrect. The full path of the file would be 281 characters long, which is more than Windows' limit of 260. But Subversion goes to a lot of trouble to avoid the path limit ... I wonder what's going on here. It seems Windows specific implementation of svn_stream__install_stream() doesn't prepend path with \\?\ for long names. Replacing svn_utf__win32_utf8_to_utf16() call with svn_io__utf8_to_unicode_longpath() should fix the problem, but I didn't try yet. Bert, is it intentional behavior or just small typo? -- Ivan Zhakov
Re: Cannot checkout or clean up using 1.9 dev build
On 29.04.2015 21:28, Bert Huijben wrote: I’m not sure if the \\?\ file://%5C%5C?%5C format is supported in the new rename api(as this is an option in certain publi apis and not a low level supported format), but it should certainly be used in the fallback codepath where the normal rename is tried, like hiw it is in apr. But if the new api fails with the limit, it is quite likely that it supports the \\?\ file://%5C%5C?%5C format. Some more recent new in Windows 8+ apis just dropped the path length limit and therefore don’t support the \\?\ file://%5C%5C?%5C format… but that doesn’t apply here as this api was new in Vista. I hope somebody looks at this earlier, but I’ll will look into this early next week. FWIW, Ivan already changed the code and added a test case, and there's a pending backport proposal in 1.9.x STATUS; the Windows buildbots seem happy with the change. -- Brane *From:* Ivan Zhakov mailto:i...@visualsvn.com *Sent:* Tuesday, April 28, 2015 9:07 AM *To:* Benjamin Fritz mailto:fritzophre...@gmail.com, Bert Huijben mailto:b...@qqmail.nl *Cc:* 'subversion' mailto:users@subversion.apache.org On 27 April 2015 at 21:35, Benjamin Fritz fritzophre...@gmail.com wrote: Apparently I'm not subscribed to get list emails; please CC me on future responses. I had to get this message from the archive :-) Branko Čibej wrote: On 27.04.2015 18:06, Benjamin Fritz wrote: I get the following error when I try to check out a working copy from my repository at work (exact path names obfuscated but length unchanged, in case it matters It probably does matter ... FWIW, I believe you did change the length of the file name, the workqueue record says 234 chars but the name in the record is 240; still, that shouldn't be an issue. Oops. You're right. My actual directory has only 3 characters where I replaced with Prog in the path string. Still, those 6 fewer characters are probably irrelevant. ), leaving a partially downloaded working copy: svn: E155009: Failed to run the WC DB work queue associated with 'C:\Project_Files\Proj_Dev\Prog_svn_1.9_cmdline', work item 2160 (file-install 234 Logical_Architecture/APP/App_PL_Verification/XXX/XXX_XXXs/XXX_7_APPDB_PROJ_PROG--0021_BE/APPDB_PROJ_PROG--0021.SUP/APPDB_PROJ_PROG--0021/APPDB_PROJ_PROG--0021/APPDB_PROJ_PROG--0021.SUP/XXX_APPDB_PROJ_PROG--002101.CRC 1 0 1 1) svn: E720123: The filename, directory name, or volume label syntax is incorrect. The full path of the file would be 281 characters long, which is more than Windows' limit of 260. But Subversion goes to a lot of trouble to avoid the path limit ... I wonder what's going on here. It seems Windows specific implementation of svn_stream__install_stream() doesn't prepend path with \\?\ for long names. Replacing svn_utf__win32_utf8_to_utf16() call with svn_io__utf8_to_unicode_longpath() should fix the problem, but I didn't try yet. Bert, is it intentional behavior or just small typo? -- Ivan Zhakov
Re: Svn rename doesn't copy custom properties
On Wed, Apr 29, 2015 at 4:01 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Dan Ellis wrote on Wed, Apr 29, 2015 at 15:43:00 -0700: Hi, We use some custom properties for tracking (e.g. my_prop:trace_to req_12345) and have noticed that svn renames do not copy these across the delete/add operation. I've tried searching the archives to see if I could find a rationale as to why, but haven't come across any. 'svn cp' and 'svn mv' do copy properties across. How exactly are you invoking those operations? Are you perhaps using a wrapper that does 'svn rm svn add'? Can you show us a transcript? On my system I see it working: [[[ % svn pl -v baz Properties on 'baz': k v % svn mv baz foo A foo D baz % svn ci -q -mm % svn pl -v foo Properties on 'foo': k v ]]] I'm guessing this could be legitimately viewed as a feature or a problem, depending on your point of view. Is there a viable option to try and get svn renames to copy these over? Including properties in mv/cp is the default behaviour and cannot be turned off (except by doing 'svn propdel' after the cp/mv and before committing). What version of the client are you using? Thanks, Dan This is specific to the rename operation (svn 1.8.9, win7): c:\Project_files\sandboxsvn pl -v .txt Properties on '.txt': myprop:trace_to req12345 c:\Project_files\sandboxsvn rename .txt .txt A .txt D .txt c:\Project_files\sandboxsvn pl -v .txt c:\Project_files\sandbox
Re: Svn rename doesn't copy custom properties
OK, so it gets stranger... I admit I changed the property names a bit to simplify them. When I ran the simplified names, it does work. Here's the exact example that does not work: c:\Project_files\sandbox_v2svn pl -v A.txt Properties on 'A.txt': pebls:plcm Test@4575 pebls:sha1 8cd8818d6b4f5edcb8b6e25cdf471af62bca403c c:\Project_files\sandbox_v2svn rename A.txt AA.txt A AA.txt D A.txt c:\Project_files\sandbox_v2svn pl -v AA.txt c:\Project_files\sandbox_v2svn pl -v REN.txt On Wed, Apr 29, 2015 at 4:13 PM, Dan Ellis danelli...@gmail.com wrote: On Wed, Apr 29, 2015 at 4:01 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Dan Ellis wrote on Wed, Apr 29, 2015 at 15:43:00 -0700: Hi, We use some custom properties for tracking (e.g. my_prop:trace_to req_12345) and have noticed that svn renames do not copy these across the delete/add operation. I've tried searching the archives to see if I could find a rationale as to why, but haven't come across any. 'svn cp' and 'svn mv' do copy properties across. How exactly are you invoking those operations? Are you perhaps using a wrapper that does 'svn rm svn add'? Can you show us a transcript? On my system I see it working: [[[ % svn pl -v baz Properties on 'baz': k v % svn mv baz foo A foo D baz % svn ci -q -mm % svn pl -v foo Properties on 'foo': k v ]]] I'm guessing this could be legitimately viewed as a feature or a problem, depending on your point of view. Is there a viable option to try and get svn renames to copy these over? Including properties in mv/cp is the default behaviour and cannot be turned off (except by doing 'svn propdel' after the cp/mv and before committing). What version of the client are you using? Thanks, Dan This is specific to the rename operation (svn 1.8.9, win7): c:\Project_files\sandboxsvn pl -v .txt Properties on '.txt': myprop:trace_to req12345 c:\Project_files\sandboxsvn rename .txt .txt A .txt D .txt c:\Project_files\sandboxsvn pl -v .txt c:\Project_files\sandbox
Svn rename doesn't copy custom properties
Hi, We use some custom properties for tracking (e.g. my_prop:trace_to req_12345) and have noticed that svn renames do not copy these across the delete/add operation. I've tried searching the archives to see if I could find a rationale as to why, but haven't come across any. I'm guessing this could be legitimately viewed as a feature or a problem, depending on your point of view. Is there a viable option to try and get svn renames to copy these over? Thanks, Dan
Re: Dealing with very old repo format (version 1)
For future reference: if svnadmin is version 1.9 or newer, it should have the 'info' subcommand, which will display both the repository format and the FS format. http://subversion.apache.org/docs/release-notes/1.9#svnadmin-info Daniel (1.9 hasn't been released yet, as of this writing) Andrew Reedick wrote on Tue, Apr 28, 2015 at 21:03:12 +: Does anyone have any tips on how to upgrade a very old repo? The db/format lists 1. A 1.8 svn client cannot hotcopy, dump or svnadmin upgrade such an old repo, all of which fail with svnadmin: E720002: Can't open file 'devel\db\current': The system cannot find the file specified. Do I need find a really old svn client (1.3?) and upgrade? Do I need to manually create the db/current file? Supposedly , a format of 1 is from pre-svn 1.0. https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_repos/repos.h - Formats 0, 1 and 2 were pre-1.0.
Re: Dealing with very old repo format (version 1)
[snipping the part about db/revs, this is only about db/revprops] Philip Martin wrote on Wed, Apr 29, 2015 at 18:53:04 +0100: Andrew Reedick jreed...@incomm.com writes: Bad News: However, it seems that I have bigger issues: * Dumped revision 109662. svnadmin: E720002: Can't open file 'devel_copy\db\revprops\109663': The system cannot find the file specified. You can replace a missing revprop file with a file containing a single line: END The file must be exactly four bytes long and must end with an LF character (0x0A). In particular, just using 'echo END foo' will work *on unix* but not on windows. Daniel
Re: Svn rename doesn't copy custom properties
Dan Ellis wrote on Wed, Apr 29, 2015 at 15:43:00 -0700: Hi, We use some custom properties for tracking (e.g. my_prop:trace_to req_12345) and have noticed that svn renames do not copy these across the delete/add operation. I've tried searching the archives to see if I could find a rationale as to why, but haven't come across any. 'svn cp' and 'svn mv' do copy properties across. How exactly are you invoking those operations? Are you perhaps using a wrapper that does 'svn rm svn add'? Can you show us a transcript? On my system I see it working: [[[ % svn pl -v baz Properties on 'baz': k v % svn mv baz foo A foo D baz % svn ci -q -mm % svn pl -v foo Properties on 'foo': k v ]]] I'm guessing this could be legitimately viewed as a feature or a problem, depending on your point of view. Is there a viable option to try and get svn renames to copy these over? Including properties in mv/cp is the default behaviour and cannot be turned off (except by doing 'svn propdel' after the cp/mv and before committing). What version of the client are you using? Thanks, Dan
Re: Svn rename doesn't copy custom properties
On Apr 29, 2015, at 4:13 PM, Dan Ellis danelli...@gmail.com wrote: This is specific to the rename operation (svn 1.8.9, win7): c:\Project_files\sandboxsvn pl -v .txt Properties on '.txt': myprop:trace_to req12345 c:\Project_files\sandboxsvn rename .txt .txt A .txt D .txt c:\Project_files\sandboxsvn pl -v .txt c:\Project_files\sandbox On OS X, svn 1.7.19, I see this after the rename: $ svn pl file1 Properties on 'file1': myprop $ svn rename file1 file2 A file2 D file1 $ svn status D file1 A +file2 $ svn pl file2 Properties on 'file2': myprop
RE: Dealing with very old repo format (version 1)
-Original Message- From: Branko Čibej [mailto:br...@wandisco.com] Are we talking about the repository format or the FSFS format here? If repos/db/fs-type says fsfs then the repository format (repos/format) is probably 3 and you're talking about repos/db/format, yes? The distinction is important. Yes, I'm referring to db/* files. $ more format fs-type :: format :: 1 :: fs-type :: fsfs In any case, 1.8 /should/ be able to dump an FSFSv1 repository, and the repos/db/current file should exists; it's been around since FSFSv1. You can try recreating it; the format is described here: https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure To find the youngest revision, look for the highest-numered file in repos/db/revs. If you're just going to dump the repository, it should be safe to set the next-node-id and next-copy-id to some large number, say 99; but I wouldn't recommend trying to commit to the repository. Please report if the above works or I'm just talking through my hat. :) -- Brane Good News: Recreating the db/current file worked in that it allowed 'svnadmin dump' to run. Bad News: However, it seems that I have bigger issues: * Dumped revision 109662. svnadmin: E720002: Can't open file 'devel_copy\db\revprops\109663': The system cannot find the file specified. When I sort the files in db/revs numerically, I see gaps in the revs: 109661 109662 109668 109734 ... 109735 157939 157940 157941 159433 159607 160600 160601 162409 And 'ls | wc -l' in dev/revs shows that there are 141,768 files, but the highest rev in db/revs is 162409... *sigh* I guess I can try piecemeal dumps. Thanks for the help everyone, but I'm thinking my missing db/current file is a symptom of the repo being mangled (probably due to inadequate backup procedures or a bad restore from tape. Rev 1 is from 2006, and the repo was just around for reference so no real worries.)
